[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:01 dist-test-slave-x07d krb5kdc[20973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:01 dist-test-slave-x07d krb5kdc[20973](info): set up 2 sockets
Sep 29 16:04:01 dist-test-slave-x07d krb5kdc[20973](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 16:04:02.683107 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.001s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:02 dist-test-slave-x07d krb5kdc[20973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:04:02.872169 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:44459
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:46827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:44459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:03.227810 20986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:03.228179 20986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:03.228350 20986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:03.235888 20986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:03.236059 20986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:03.236161 20986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:03.236264 20986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:03.236338 20986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:03.247327 20986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:46827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:44459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:44459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:03.248804 20986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:03.250409 20986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 16:04:03.264796 20986 server_base.cc:1034] running on GCE node
W20260929 16:04:03.264776 20991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:03.265098 20994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:03.265264 20992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:03.266573 20986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:03.268175 20986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:03.269389 20986 hybrid_clock.cc:648] HybridClock initialized: now 1790697843269332 us; error 40 us; skew 500 ppm
I20260929 16:04:03.269798 20986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:03 dist-test-slave-x07d krb5kdc[20973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697843, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:03.273379 20986 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:04:03.275134 20986 webserver.cc:460] Webserver started at http://127.20.121.190:39027/ using document root <none> and password file <none>
I20260929 16:04:03.275909 20986 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:03.276064 20986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:03.276388 20986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:03.279692 20986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "6e48d4631b83453a8b5ac9352c5a91d1"
format_stamp: "Formatted at 2026-09-29 16:04:03 on dist-test-slave-x07d"
server_key: "2c45b257acecad787c0906c7f0f77428"
server_key_iv: "4355c7a6053602bcf9364618d8eb7a19"
server_key_version: "encryptionkey@0"
I20260929 16:04:03.280526 20986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "6e48d4631b83453a8b5ac9352c5a91d1"
format_stamp: "Formatted at 2026-09-29 16:04:03 on dist-test-slave-x07d"
server_key: "2c45b257acecad787c0906c7f0f77428"
server_key_iv: "4355c7a6053602bcf9364618d8eb7a19"
server_key_version: "encryptionkey@0"
I20260929 16:04:03.286228 20986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260929 16:04:03.290499 21001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:03.291930 20986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 16:04:03.292177 20986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "6e48d4631b83453a8b5ac9352c5a91d1"
format_stamp: "Formatted at 2026-09-29 16:04:03 on dist-test-slave-x07d"
server_key: "2c45b257acecad787c0906c7f0f77428"
server_key_iv: "4355c7a6053602bcf9364618d8eb7a19"
server_key_version: "encryptionkey@0"
I20260929 16:04:03.292392 20986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:03.318500 20986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:03.323218 20986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:03.323550 20986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:03.344655 20986 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:44459
I20260929 16:04:03.344687 21052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:44459 every 8 connection(s)
I20260929 16:04:03.346113 20986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:03.350515 21053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:03.351753 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 20986
I20260929 16:04:03.352110 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:03.355240 20966 external_mini_cluster.cc:1428] Setting key 066f987d86c6875256232ceddadd5e02
I20260929 16:04:03.365420 21053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1: Bootstrap starting.
I20260929 16:04:03.369295 21053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:03.370875 21053 log.cc:824] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:03.374677 21053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1: No bootstrap required, opened a new log
I20260929 16:04:03.385200 21053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e48d4631b83453a8b5ac9352c5a91d1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44459 } }
I20260929 16:04:03.385644 21053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:03.385766 21053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e48d4631b83453a8b5ac9352c5a91d1, State: Initialized, Role: FOLLOWER
I20260929 16:04:03.386530 21053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e48d4631b83453a8b5ac9352c5a91d1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44459 } }
I20260929 16:04:03.386781 21053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:03.386947 21053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:03.387180 21053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 16:04:03 dist-test-slave-x07d krb5kdc[20973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:03.390430 21053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e48d4631b83453a8b5ac9352c5a91d1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44459 } }
I20260929 16:04:03.391162 21053 leader_election.cc:302] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e48d4631b83453a8b5ac9352c5a91d1; no voters:
I20260929 16:04:03.391959 21053 leader_election.cc:288] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:03.392066 21058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:03.394927 21053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:03.395097 21058 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [term 1 LEADER]: Becoming Leader. State: Replica: 6e48d4631b83453a8b5ac9352c5a91d1, State: Running, Role: LEADER
I20260929 16:04:03.395709 21058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e48d4631b83453a8b5ac9352c5a91d1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44459 } }
I20260929 16:04:03.403469 21059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e48d4631b83453a8b5ac9352c5a91d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e48d4631b83453a8b5ac9352c5a91d1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44459 } } }
I20260929 16:04:03.403894 21059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:03.404601 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:03.360503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.20.121.190:44459)
0929 16:04:03.361466 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:03.361495 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:03.362895 (+ 1400us) server_negotiation.cc:407] Connection header received
0929 16:04:03.372540 (+ 9645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:03.372576 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:03.373152 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:03.373696 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:03.376892 (+ 3196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:03.379321 (+ 2429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:03.381150 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:03.382410 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:03.382655 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:03.389076 (+ 6421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:03.389091 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:03.389118 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:03.389179 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:03.394046 (+ 4867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:03.395390 (+ 1344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:03.395398 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:03.395408 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:03.395661 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:03.396269 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:03.396274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:03.396278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:03.397372 (+ 1094us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:03.401101 (+ 3729us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:03.401515 (+ 414us) server_negotiation.cc:299] Negotiation successful
0929 16:04:03.402114 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":385,"threads_started":1}
I20260929 16:04:03.404451 21060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e48d4631b83453a8b5ac9352c5a91d1. Latest consensus state: current_term: 1 leader_uuid: "6e48d4631b83453a8b5ac9352c5a91d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e48d4631b83453a8b5ac9352c5a91d1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44459 } } }
I20260929 16:04:03.405357 21060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:03.425511 21068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:03.432076 21068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:03.443032 21068 catalog_manager.cc:1347] Generated new cluster ID: 815b5172c5dc45a3bd28454c25b2b17b
I20260929 16:04:03.443224 21068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:03.458001 21068 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:03.459432 21068 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:03.469246 21068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e48d4631b83453a8b5ac9352c5a91d1: Generated new TSK 0
I20260929 16:04:03.470260 21068 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:04:03.527611 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44459
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:46827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:03.900000 21080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:03.900287 21080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:03.900382 21080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:03.908771 21080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:03.908890 21080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:03.909005 21080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:03.918936 21080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:46827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:03.920241 21080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:03.921567 21080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:03.936125 21085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:03.936074 21086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:03.936007 21088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:03.937361 21080 server_base.cc:1034] running on GCE node
I20260929 16:04:03.938117 21080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:03.939132 21080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:03.940366 21080 hybrid_clock.cc:648] HybridClock initialized: now 1790697843940318 us; error 35 us; skew 500 ppm
I20260929 16:04:03.940729 21080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:03 dist-test-slave-x07d krb5kdc[20973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697843, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:03.944082 21080 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:04:03.945780 21080 webserver.cc:460] Webserver started at http://127.20.121.129:41047/ using document root <none> and password file <none>
I20260929 16:04:03.946535 21080 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:03.946637 21080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:03.946897 21080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:03.949836 21080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "359efabd5cef4802a03da2a8150d4e36"
format_stamp: "Formatted at 2026-09-29 16:04:03 on dist-test-slave-x07d"
server_key: "3221749157ef7554845165af2908f3df"
server_key_iv: "dc0b9099395db1c466a0042537717429"
server_key_version: "encryptionkey@0"
I20260929 16:04:03.950716 21080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "359efabd5cef4802a03da2a8150d4e36"
format_stamp: "Formatted at 2026-09-29 16:04:03 on dist-test-slave-x07d"
server_key: "3221749157ef7554845165af2908f3df"
server_key_iv: "dc0b9099395db1c466a0042537717429"
server_key_version: "encryptionkey@0"
I20260929 16:04:03.956655 21080 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 16:04:03.960414 21095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:03.961935 21080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:04:03.962183 21080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "359efabd5cef4802a03da2a8150d4e36"
format_stamp: "Formatted at 2026-09-29 16:04:03 on dist-test-slave-x07d"
server_key: "3221749157ef7554845165af2908f3df"
server_key_iv: "dc0b9099395db1c466a0042537717429"
server_key_version: "encryptionkey@0"
I20260929 16:04:03.962402 21080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:03.993270 21080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:03.998600 21080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:03.998905 21080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:03.999967 21080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:04.001423 21080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:04.001533 21080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:04.001695 21080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:04.001776 21080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:04.038555 21080 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:43783
I20260929 16:04:04.038600 21207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:43783 every 8 connection(s)
I20260929 16:04:04.040030 21080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:04.047359 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 21080
I20260929 16:04:04.047655 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:04.048519 20966 external_mini_cluster.cc:1428] Setting key 180b5ebb7dc55f7eae7b4f850322d9f5
Sep 29 16:04:04 dist-test-slave-x07d krb5kdc[20973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697843, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:04:04.066573 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:04.043410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:51147 (local address 127.20.121.190:44459)
0929 16:04:04.044123 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:04.044128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:04.045109 (+ 981us) server_negotiation.cc:407] Connection header received
0929 16:04:04.047516 (+ 2407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:04.047525 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:04.047625 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:04.047758 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:04.050252 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:04.051170 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:04.054155 (+ 2985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:04.054549 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:04.054666 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:04.058782 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:04.058791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:04.058799 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:04.058845 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:04.062784 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:04.063710 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:04.063718 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:04.063725 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:04.063887 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:04.064369 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:04.064376 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:04.064380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:04.064781 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:04.064941 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:04.065773 (+ 832us) server_negotiation.cc:299] Negotiation successful
0929 16:04:04.065998 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":362,"threads_started":1}
I20260929 16:04:04.067477 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:04.044139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:44459 (local address 127.20.121.129:51147)
0929 16:04:04.044892 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:04.044955 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:04.047120 (+ 2165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:04.048085 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:04.048101 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:04.048967 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:04.050004 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:04.050077 (+ 73us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:04.051344 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:04.051353 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:04.053957 (+ 2604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:04.053977 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:04.054803 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:04.054813 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:04.055206 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:04.056136 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:04.056161 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:04.058472 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:04.063027 (+ 4555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:04.063035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:04.063050 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:04.063554 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:04.064036 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:04.064040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:04.064042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:04.064238 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:04.064986 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:04.064993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:04.065359 (+ 366us) client_negotiation.cc:769] Sending connection context
0929 16:04:04.065614 (+ 255us) client_negotiation.cc:240] Negotiation successful
0929 16:04:04.066102 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":317,"threads_started":1}
I20260929 16:04:04.069236 21208 heartbeater.cc:347] Connected to a master server at 127.20.121.190:44459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:04:04.069613 21208 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:04.070865 21208 heartbeater.cc:511] Master 127.20.121.190:44459 requested a full tablet report, sending...
I20260929 16:04:04.073436 21018 ts_manager.cc:194] Registered new tserver with Master: 359efabd5cef4802a03da2a8150d4e36 (127.20.121.129:43783)
I20260929 16:04:04.075080 21018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:51147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:04:04.095018 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44459
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:46827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:04.446206 21214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:04.446584 21214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:04.446758 21214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:04.454521 21214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:04.454712 21214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:04.454908 21214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:04.465747 21214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:46827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:04.467448 21214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:04.468842 21214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:04.483458 21219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:04.483525 21222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:04.483831 21220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:04.484755 21214 server_base.cc:1034] running on GCE node
I20260929 16:04:04.485564 21214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:04.486735 21214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:04.488022 21214 hybrid_clock.cc:648] HybridClock initialized: now 1790697844487964 us; error 47 us; skew 500 ppm
I20260929 16:04:04.488476 21214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:04 dist-test-slave-x07d krb5kdc[20973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697844, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:04.492301 21214 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:04:04.494122 21214 webserver.cc:460] Webserver started at http://127.20.121.130:41245/ using document root <none> and password file <none>
I20260929 16:04:04.494880 21214 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:04.495044 21214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:04.495430 21214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:04.499496 21214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "5f24586d48d34154aa00a08095476fca"
format_stamp: "Formatted at 2026-09-29 16:04:04 on dist-test-slave-x07d"
server_key: "2ce821fac40ba162106c27ab0b7c4b7f"
server_key_iv: "1a98714b8217df1729059f005224af62"
server_key_version: "encryptionkey@0"
I20260929 16:04:04.500501 21214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "5f24586d48d34154aa00a08095476fca"
format_stamp: "Formatted at 2026-09-29 16:04:04 on dist-test-slave-x07d"
server_key: "2ce821fac40ba162106c27ab0b7c4b7f"
server_key_iv: "1a98714b8217df1729059f005224af62"
server_key_version: "encryptionkey@0"
I20260929 16:04:04.507218 21214 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260929 16:04:04.512015 21229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:04.513703 21214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:04:04.513897 21214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "5f24586d48d34154aa00a08095476fca"
format_stamp: "Formatted at 2026-09-29 16:04:04 on dist-test-slave-x07d"
server_key: "2ce821fac40ba162106c27ab0b7c4b7f"
server_key_iv: "1a98714b8217df1729059f005224af62"
server_key_version: "encryptionkey@0"
I20260929 16:04:04.514235 21214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:04.540778 21214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:04.545888 21214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:04.546332 21214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:04.547503 21214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:04.549084 21214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:04.549172 21214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:04.549312 21214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:04.549379 21214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:04.592775 21214 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:35251
I20260929 16:04:04.592839 21341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:35251 every 8 connection(s)
I20260929 16:04:04.594400 21214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:04.594920 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 21214
I20260929 16:04:04.595132 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:04.595825 20966 external_mini_cluster.cc:1428] Setting key 06c20bd0ee218b483a460d8121566155
Sep 29 16:04:04 dist-test-slave-x07d krb5kdc[20973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697844, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
I20260929 16:04:04.620469 21344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:04.597706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:38081 (local address 127.20.121.190:44459)
0929 16:04:04.598265 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:04.598272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:04.599838 (+ 1566us) server_negotiation.cc:407] Connection header received
0929 16:04:04.601885 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:04.601891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:04.601987 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:04.602153 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:04.603995 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:04.604783 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:04.606626 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:04.607087 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:04.607249 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:04.612289 (+ 5040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:04.612294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:04.612298 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:04.612337 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:04.615823 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:04.616844 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:04.616858 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:04.616860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:04.617020 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:04.617585 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:04.617590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:04.617592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:04.617934 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:04.618166 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:04.619192 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0929 16:04:04.619485 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":290,"threads_started":1}
I20260929 16:04:04.621289 21345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:04.598548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:44459 (local address 127.20.121.130:38081)
0929 16:04:04.599616 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:04.599682 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:04.601547 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:04.602474 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:04.602484 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:04.603129 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:04.603787 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:04.603808 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:04.604964 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:04.604977 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:04.606443 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:04.606458 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:04.607376 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:04.607383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:04.608118 (+ 735us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:04.609068 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:04.609096 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:04.611967 (+ 2871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:04.616048 (+ 4081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:04.616062 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:04.616092 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:04.616661 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:04.617173 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:04.617178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:04.617182 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:04.617423 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:04.618146 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:04.618156 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:04.618732 (+ 576us) client_negotiation.cc:769] Sending connection context
0929 16:04:04.619089 (+ 357us) client_negotiation.cc:240] Negotiation successful
0929 16:04:04.619737 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":505,"threads_started":1}
I20260929 16:04:04.623342 21342 heartbeater.cc:347] Connected to a master server at 127.20.121.190:44459
I20260929 16:04:04.623740 21342 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:04.624657 21342 heartbeater.cc:511] Master 127.20.121.190:44459 requested a full tablet report, sending...
I20260929 16:04:04.626533 21018 ts_manager.cc:194] Registered new tserver with Master: 5f24586d48d34154aa00a08095476fca (127.20.121.130:35251)
I20260929 16:04:04.627883 21018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:38081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:04:04.643143 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44459
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:46827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:05.004707 21348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:05.005030 21348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:05.005195 21348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:05.013316 21348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:05.013453 21348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:05.013645 21348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:05.025313 21348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:46827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:05.026919 21348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:05.028450 21348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:05.044798 21353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:05.045048 21356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:05.045554 21354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:05.046729 21348 server_base.cc:1034] running on GCE node
I20260929 16:04:05.047451 21348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:05.048611 21348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:05.049836 21348 hybrid_clock.cc:648] HybridClock initialized: now 1790697845049788 us; error 49 us; skew 500 ppm
I20260929 16:04:05.050226 21348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[20973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697845, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:05.053913 21348 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:04:05.055644 21348 webserver.cc:460] Webserver started at http://127.20.121.131:36425/ using document root <none> and password file <none>
I20260929 16:04:05.056666 21348 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:05.056836 21348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:05.057593 21348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:05.060802 21348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "4cec1fc367704dbca47bcc89cb684ca5"
format_stamp: "Formatted at 2026-09-29 16:04:05 on dist-test-slave-x07d"
server_key: "64fb5e40430d4a7cc6c753616cfad5d2"
server_key_iv: "1e073b81c62edc03b42ed2f03c9337b3"
server_key_version: "encryptionkey@0"
I20260929 16:04:05.062081 21348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "4cec1fc367704dbca47bcc89cb684ca5"
format_stamp: "Formatted at 2026-09-29 16:04:05 on dist-test-slave-x07d"
server_key: "64fb5e40430d4a7cc6c753616cfad5d2"
server_key_iv: "1e073b81c62edc03b42ed2f03c9337b3"
server_key_version: "encryptionkey@0"
I20260929 16:04:05.068728 21348 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260929 16:04:05.073261 21363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:05.074941 21348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:05.075160 21348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "4cec1fc367704dbca47bcc89cb684ca5"
format_stamp: "Formatted at 2026-09-29 16:04:05 on dist-test-slave-x07d"
server_key: "64fb5e40430d4a7cc6c753616cfad5d2"
server_key_iv: "1e073b81c62edc03b42ed2f03c9337b3"
server_key_version: "encryptionkey@0"
I20260929 16:04:05.075397 21348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:05.078475 21208 heartbeater.cc:503] Master 127.20.121.190:44459 was elected leader, sending a full tablet report...
I20260929 16:04:05.090879 21348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:05.097339 21348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:05.097635 21348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:05.098840 21348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:05.100306 21348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:05.100383 21348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:05.100519 21348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:05.100569 21348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:05.139456 21348 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:33443
I20260929 16:04:05.139556 21475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:33443 every 8 connection(s)
I20260929 16:04:05.140863 21348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:05.142661 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 21348
I20260929 16:04:05.142885 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:05.143548 20966 external_mini_cluster.cc:1428] Setting key 4ed1746a69276056eced794b46d0fff8
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[20973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697845, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:05.163574 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:05.144157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:39673 (local address 127.20.121.190:44459)
0929 16:04:05.144663 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:05.144669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:05.145922 (+ 1253us) server_negotiation.cc:407] Connection header received
0929 16:04:05.148241 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:05.148246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:05.148320 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:05.148417 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:05.149985 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.150915 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.152500 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.152884 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.152984 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:05.156582 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:05.156587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:05.156591 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:05.156626 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:05.160075 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.161046 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.161051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.161053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.161190 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.161683 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.161687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.161689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.162114 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:05.162250 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:05.163078 (+ 828us) server_negotiation.cc:299] Negotiation successful
0929 16:04:05.163266 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":232,"threads_started":1}
I20260929 16:04:05.164772 21479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:05.144850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:44459 (local address 127.20.121.131:39673)
0929 16:04:05.145756 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:05.145808 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:05.147968 (+ 2160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:05.148670 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:05.148679 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:05.149209 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:05.149806 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:05.149825 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.151071 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.151075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:05.152355 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:05.152371 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.153080 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.153085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:05.153291 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:05.153975 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:05.153997 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:05.156341 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:05.160258 (+ 3917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:05.160269 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:05.160289 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:05.160918 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:05.161308 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:05.161312 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:05.161314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:05.161553 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:05.162264 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:05.162272 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:05.162670 (+ 398us) client_negotiation.cc:769] Sending connection context
0929 16:04:05.162978 (+ 308us) client_negotiation.cc:240] Negotiation successful
0929 16:04:05.163477 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":357,"threads_started":1}
I20260929 16:04:05.166524 21476 heartbeater.cc:347] Connected to a master server at 127.20.121.190:44459
I20260929 16:04:05.167017 21476 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:05.167826 21476 heartbeater.cc:511] Master 127.20.121.190:44459 requested a full tablet report, sending...
I20260929 16:04:05.169447 21018 ts_manager.cc:194] Registered new tserver with Master: 4cec1fc367704dbca47bcc89cb684ca5 (127.20.121.131:33443)
I20260929 16:04:05.170465 21018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:39673
I20260929 16:04:05.173312 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[20973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697845, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[20973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697845, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:05.221206 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:05.204325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.20.121.129:43783)
0929 16:04:05.204989 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:05.204996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:05.205024 (+ 28us) server_negotiation.cc:407] Connection header received
0929 16:04:05.205113 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:05.205121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:05.205344 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:05.205467 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:05.207101 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.208142 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.209578 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.210396 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.210528 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:05.214708 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:05.214723 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:05.214739 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:05.214780 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:05.217942 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.218742 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.218750 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.218753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.219009 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.219491 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.219498 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.219503 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.219908 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:05.220205 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:05.220536 (+ 331us) server_negotiation.cc:299] Negotiation successful
0929 16:04:05.220760 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":345,"threads_started":1}
W20260929 16:04:05.222612 21123 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55506
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[20973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[20973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:05.269107 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:05.253743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.20.121.129:43783)
0929 16:04:05.253967 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:05.253973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:05.254334 (+ 361us) server_negotiation.cc:407] Connection header received
0929 16:04:05.254491 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:05.254496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:05.254594 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:05.254689 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:05.256459 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.257176 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.258629 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.259060 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.259184 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:05.263404 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:05.263410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:05.263414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:05.263455 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:05.266272 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.267090 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.267094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.267097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.267274 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.267746 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.267751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.267753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.268131 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:05.268337 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:05.268611 (+ 274us) server_negotiation.cc:299] Negotiation successful
0929 16:04:05.268781 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 16:04:05.270722 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 21080
I20260929 16:04:05.285178 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:05.271763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.20.121.129:43783)
0929 16:04:05.272004 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:05.272012 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:05.272290 (+ 278us) server_negotiation.cc:407] Connection header received
0929 16:04:05.272479 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:05.272483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:05.272557 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:05.272692 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:05.273875 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.274646 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.276408 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.276859 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.276995 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:05.279751 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:05.279758 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:05.279761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:05.279799 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:05.282562 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.283269 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.283276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.283280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.283422 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.283776 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.283782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.283786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.284046 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:05.284296 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:05.284584 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 16:04:05.284805 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:04:05.285919 21203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:05.380913 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 21080
I20260929 16:04:05.403380 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 21214
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[20973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:04:05.420502 21491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:05.404268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46854 (local address 127.20.121.130:35251)
0929 16:04:05.404774 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:05.404780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:05.404804 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:04:05.404861 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:05.404868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:05.405078 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:05.405252 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:05.406505 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.407499 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.408785 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.409373 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.409518 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:05.414180 (+ 4662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:05.414196 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:05.414215 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:05.414260 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:05.417346 (+ 3086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.418235 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.418243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.418245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.418379 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.418831 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.418835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.418838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.419221 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:05.419577 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:05.419850 (+ 273us) server_negotiation.cc:299] Negotiation successful
0929 16:04:05.420131 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":249,"threads_started":1}
I20260929 16:04:05.421932 21337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:05.507680 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 21214
I20260929 16:04:05.524976 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 21348
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[20973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:04:05.542847 21493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:05.526168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.20.121.131:33443)
0929 16:04:05.526820 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:05.526827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:05.526856 (+ 29us) server_negotiation.cc:407] Connection header received
0929 16:04:05.526920 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:05.526926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:05.527147 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:05.527331 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:05.528642 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.529691 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.531101 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:05.531700 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:05.531862 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:05.536701 (+ 4839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:05.536718 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:05.536735 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:05.536779 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:05.539957 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.540798 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.540807 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.540809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.540944 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:05.541323 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:05.541327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:05.541329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:05.541705 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:05.541943 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:05.542336 (+ 393us) server_negotiation.cc:299] Negotiation successful
0929 16:04:05.542547 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":323,"threads_started":1}
I20260929 16:04:05.544292 21471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:05.630812 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 21348
I20260929 16:04:05.647822 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 20986
I20260929 16:04:05.649430 21048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:05.755879 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 20986
2026-09-29T16:04:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4255 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[21498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[21498](info): set up 2 sockets
Sep 29 16:04:05 dist-test-slave-x07d krb5kdc[21498](info): commencing operation
krb5kdc: starting...
W20260929 16:04:06.906152 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:07 dist-test-slave-x07d krb5kdc[21498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:04:07.094791 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38563
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:38563
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:07.426120 21511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:07.426440 21511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:07.426604 21511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:07.433979 21511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:07.434193 21511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:07.434293 21511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:07.434396 21511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:07.434478 21511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:07.444742 21511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:38563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:07.446254 21511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:07.447688 21511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:07.460698 21519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:07.460742 21516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:07.460819 21517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:07.461596 21511 server_base.cc:1034] running on GCE node
I20260929 16:04:07.462570 21511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:07.464289 21511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:07.465564 21511 hybrid_clock.cc:648] HybridClock initialized: now 1790697847465501 us; error 46 us; skew 500 ppm
I20260929 16:04:07.465998 21511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:07 dist-test-slave-x07d krb5kdc[21498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697847, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:07.476373 21511 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:04:07.478286 21511 webserver.cc:460] Webserver started at http://127.20.121.190:41933/ using document root <none> and password file <none>
I20260929 16:04:07.479074 21511 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:07.479192 21511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:07.479543 21511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:07.483222 21511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "f9edfcb3817341b6859fadd87213ff24"
format_stamp: "Formatted at 2026-09-29 16:04:07 on dist-test-slave-x07d"
server_key: "ea4b2c4983ed26d7e57672cbed1c6fe4"
server_key_iv: "07e34ed9cfdb911e384b3e7e6b142a71"
server_key_version: "encryptionkey@0"
I20260929 16:04:07.484100 21511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "f9edfcb3817341b6859fadd87213ff24"
format_stamp: "Formatted at 2026-09-29 16:04:07 on dist-test-slave-x07d"
server_key: "ea4b2c4983ed26d7e57672cbed1c6fe4"
server_key_iv: "07e34ed9cfdb911e384b3e7e6b142a71"
server_key_version: "encryptionkey@0"
I20260929 16:04:07.490332 21511 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260929 16:04:07.494566 21526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:07.496332 21511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 16:04:07.496690 21511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "f9edfcb3817341b6859fadd87213ff24"
format_stamp: "Formatted at 2026-09-29 16:04:07 on dist-test-slave-x07d"
server_key: "ea4b2c4983ed26d7e57672cbed1c6fe4"
server_key_iv: "07e34ed9cfdb911e384b3e7e6b142a71"
server_key_version: "encryptionkey@0"
I20260929 16:04:07.496956 21511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:07.511327 21511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:07.516004 21511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:07.516338 21511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:07.536795 21511 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:38563
I20260929 16:04:07.536798 21577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:38563 every 8 connection(s)
I20260929 16:04:07.538352 21511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:07.542478 21578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:07.543140 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 21511
I20260929 16:04:07.543397 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:07.544319 20966 external_mini_cluster.cc:1428] Setting key c0610663a9c70cfdcf5c58e1c73645ce
I20260929 16:04:07.558482 21578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24: Bootstrap starting.
Sep 29 16:04:07 dist-test-slave-x07d krb5kdc[21498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:07.562390 21578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:07.563648 21578 log.cc:824] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:07.566800 21578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24: No bootstrap required, opened a new log
I20260929 16:04:07.568879 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:07.546547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.20.121.190:38563)
0929 16:04:07.547358 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:07.547373 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:07.547415 (+ 42us) server_negotiation.cc:407] Connection header received
0929 16:04:07.549447 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:07.549477 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:07.549913 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:07.550369 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:07.552322 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:07.553690 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:07.555677 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:07.556383 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:07.556514 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:07.561609 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:07.561617 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:07.561634 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:07.561680 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:07.564861 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:07.565562 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:07.565571 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:07.565577 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:07.565735 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:07.566143 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:07.566148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:07.566150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:07.566656 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:07.566959 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:07.567453 (+ 494us) server_negotiation.cc:299] Negotiation successful
0929 16:04:07.567856 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":353,"threads_started":1}
I20260929 16:04:07.575459 21578 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9edfcb3817341b6859fadd87213ff24" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38563 } }
I20260929 16:04:07.575925 21578 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:07.576049 21578 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9edfcb3817341b6859fadd87213ff24, State: Initialized, Role: FOLLOWER
I20260929 16:04:07.576689 21578 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9edfcb3817341b6859fadd87213ff24" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38563 } }
I20260929 16:04:07.576963 21578 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:07.577117 21578 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:07.577327 21578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:07.580149 21578 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9edfcb3817341b6859fadd87213ff24" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38563 } }
I20260929 16:04:07.580724 21578 leader_election.cc:302] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9edfcb3817341b6859fadd87213ff24; no voters:
I20260929 16:04:07.581367 21578 leader_election.cc:288] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:07.581533 21583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:07.582902 21583 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [term 1 LEADER]: Becoming Leader. State: Replica: f9edfcb3817341b6859fadd87213ff24, State: Running, Role: LEADER
I20260929 16:04:07.583503 21583 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9edfcb3817341b6859fadd87213ff24" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38563 } }
I20260929 16:04:07.583932 21578 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:07.586476 21584 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9edfcb3817341b6859fadd87213ff24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9edfcb3817341b6859fadd87213ff24" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38563 } } }
I20260929 16:04:07.586447 21585 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9edfcb3817341b6859fadd87213ff24. Latest consensus state: current_term: 1 leader_uuid: "f9edfcb3817341b6859fadd87213ff24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9edfcb3817341b6859fadd87213ff24" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38563 } } }
I20260929 16:04:07.586939 21584 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:07.586939 21585 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:07.588132 21591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:07.596246 21591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:07.607329 21591 catalog_manager.cc:1347] Generated new cluster ID: 64dfcb37e4334d0e98f05cceb876aa4c
I20260929 16:04:07.607479 21591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:07.618999 21591 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:07.620955 21591 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:07.632877 21591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9edfcb3817341b6859fadd87213ff24: Generated new TSK 0
I20260929 16:04:07.634155 21591 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:04:07.683434 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38563
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:08.039412 21604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:08.039753 21604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:08.039933 21604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:08.047474 21604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:08.047616 21604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:08.047788 21604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:08.057823 21604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38563
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:08.059294 21604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:08.060741 21604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:08.074860 21609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:08.074991 21610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:08.074883 21612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:08.075678 21604 server_base.cc:1034] running on GCE node
I20260929 16:04:08.076375 21604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:08.077378 21604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:08.078626 21604 hybrid_clock.cc:648] HybridClock initialized: now 1790697848078561 us; error 60 us; skew 500 ppm
I20260929 16:04:08.078985 21604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:08 dist-test-slave-x07d krb5kdc[21498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697848, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:08.082288 21604 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:04:08.083776 21604 webserver.cc:460] Webserver started at http://127.20.121.129:35759/ using document root <none> and password file <none>
I20260929 16:04:08.084479 21604 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:08.084609 21604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:08.084944 21604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:08.088129 21604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb"
format_stamp: "Formatted at 2026-09-29 16:04:08 on dist-test-slave-x07d"
server_key: "aa5a94d70327001bbf0843644e557a59"
server_key_iv: "7361fbf1e5811df4c4aec4673afef385"
server_key_version: "encryptionkey@0"
I20260929 16:04:08.088928 21604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb"
format_stamp: "Formatted at 2026-09-29 16:04:08 on dist-test-slave-x07d"
server_key: "aa5a94d70327001bbf0843644e557a59"
server_key_iv: "7361fbf1e5811df4c4aec4673afef385"
server_key_version: "encryptionkey@0"
I20260929 16:04:08.094940 21604 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260929 16:04:08.099311 21619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:08.100970 21604 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260929 16:04:08.101169 21604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb"
format_stamp: "Formatted at 2026-09-29 16:04:08 on dist-test-slave-x07d"
server_key: "aa5a94d70327001bbf0843644e557a59"
server_key_iv: "7361fbf1e5811df4c4aec4673afef385"
server_key_version: "encryptionkey@0"
I20260929 16:04:08.101428 21604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:08.120138 21604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:08.124694 21604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:08.125007 21604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:08.126078 21604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:08.127507 21604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:08.127585 21604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:08.127715 21604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:08.127775 21604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:08.169224 21604 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:38921
I20260929 16:04:08.169274 21731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:38921 every 8 connection(s)
I20260929 16:04:08.170718 21604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:08.172276 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 21604
I20260929 16:04:08.172505 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:08.173209 20966 external_mini_cluster.cc:1428] Setting key 8070befd290d2a319522694e647f5073
Sep 29 16:04:08 dist-test-slave-x07d krb5kdc[21498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697848, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:04:08.195438 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:08.173641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:48023 (local address 127.20.121.190:38563)
0929 16:04:08.174148 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:08.174154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:08.175628 (+ 1474us) server_negotiation.cc:407] Connection header received
0929 16:04:08.177553 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:08.177566 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:08.177662 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:08.177791 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:08.179566 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:08.180316 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:08.183346 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:08.183885 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:08.183991 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:08.188628 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:08.188636 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:08.188641 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:08.188689 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:08.192023 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:08.192783 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:08.192788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:08.192790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:08.192909 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:08.193580 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:08.193583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:08.193585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:08.193886 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:08.194182 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:08.194990 (+ 808us) server_negotiation.cc:299] Negotiation successful
0929 16:04:08.195174 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":233,"threads_started":1}
I20260929 16:04:08.197172 21735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:08.174566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38563 (local address 127.20.121.129:48023)
0929 16:04:08.175474 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:08.175523 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:08.177262 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:08.178123 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:08.178132 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:08.178662 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:08.179346 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:08.179373 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:08.180488 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:08.180496 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:08.183131 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:08.183154 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:08.184153 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:08.184162 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:08.184517 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:08.185337 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:08.185380 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:08.188334 (+ 2954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:08.192179 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:08.192187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:08.192201 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:08.192637 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:08.193019 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:08.193022 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:08.193024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:08.193434 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:08.194059 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:08.194071 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:08.194536 (+ 465us) client_negotiation.cc:769] Sending connection context
0929 16:04:08.194888 (+ 352us) client_negotiation.cc:240] Negotiation successful
0929 16:04:08.195519 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":379,"threads_started":1}
I20260929 16:04:08.198953 21732 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38563
I20260929 16:04:08.199316 21732 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:08.200337 21732 heartbeater.cc:511] Master 127.20.121.190:38563 requested a full tablet report, sending...
I20260929 16:04:08.202785 21543 ts_manager.cc:194] Registered new tserver with Master: 2808e14ea0cf4a1fa134cf3f97fc10eb (127.20.121.129:38921)
I20260929 16:04:08.204310 21543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:48023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:04:08.222718 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38563
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:08.566955 21738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:08.567325 21738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:08.567495 21738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:08.575273 21738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:08.575454 21738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:08.575665 21738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:08.586292 21738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38563
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:08.588181 21738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:08.589701 21738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:08.608886 21743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:08.608812 21746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:08.608793 21744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:08.609745 21738 server_base.cc:1034] running on GCE node
I20260929 16:04:08.610643 21738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:08.611867 21738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:08.613197 21738 hybrid_clock.cc:648] HybridClock initialized: now 1790697848613142 us; error 54 us; skew 500 ppm
I20260929 16:04:08.613627 21738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:08 dist-test-slave-x07d krb5kdc[21498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697848, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:08.617652 21738 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:04:08.619731 21738 webserver.cc:460] Webserver started at http://127.20.121.130:36343/ using document root <none> and password file <none>
I20260929 16:04:08.620621 21738 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:08.620779 21738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:08.621134 21738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:08.624433 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "f2a925bc8632485f833e806a83b23a0c"
format_stamp: "Formatted at 2026-09-29 16:04:08 on dist-test-slave-x07d"
server_key: "1f48c995359f6cd5e0a95bd360687456"
server_key_iv: "919ba08d1d5f198154c5a6308bd021bd"
server_key_version: "encryptionkey@0"
I20260929 16:04:08.625365 21738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "f2a925bc8632485f833e806a83b23a0c"
format_stamp: "Formatted at 2026-09-29 16:04:08 on dist-test-slave-x07d"
server_key: "1f48c995359f6cd5e0a95bd360687456"
server_key_iv: "919ba08d1d5f198154c5a6308bd021bd"
server_key_version: "encryptionkey@0"
I20260929 16:04:08.631467 21738 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260929 16:04:08.635512 21753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:08.637130 21738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:04:08.637336 21738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "f2a925bc8632485f833e806a83b23a0c"
format_stamp: "Formatted at 2026-09-29 16:04:08 on dist-test-slave-x07d"
server_key: "1f48c995359f6cd5e0a95bd360687456"
server_key_iv: "919ba08d1d5f198154c5a6308bd021bd"
server_key_version: "encryptionkey@0"
I20260929 16:04:08.637550 21738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:08.666220 21738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:08.671015 21738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:08.671341 21738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:08.672421 21738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:08.673914 21738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:08.673995 21738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:08.674161 21738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:08.674211 21738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:08.709615 21738 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:45967
I20260929 16:04:08.709661 21865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:45967 every 8 connection(s)
I20260929 16:04:08.710994 21738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:08.712865 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 21738
I20260929 16:04:08.713088 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:08.713661 20966 external_mini_cluster.cc:1428] Setting key 3562e3bf1fb546ffca8371f94a425e7c
Sep 29 16:04:08 dist-test-slave-x07d krb5kdc[21498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697848, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
I20260929 16:04:08.736230 21868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:08.714535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:58955 (local address 127.20.121.190:38563)
0929 16:04:08.715270 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:08.715276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:08.716856 (+ 1580us) server_negotiation.cc:407] Connection header received
0929 16:04:08.718976 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:08.718984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:08.719077 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:08.719201 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:08.720849 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:08.722090 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:08.723704 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:08.724485 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:08.724634 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:08.728908 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:08.728915 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:08.728920 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:08.728964 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:08.732197 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:08.733024 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:08.733029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:08.733032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:08.733184 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:08.733763 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:08.733768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:08.733770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:08.734283 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:08.734472 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:08.735291 (+ 819us) server_negotiation.cc:299] Negotiation successful
0929 16:04:08.735659 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":355,"threads_started":1}
I20260929 16:04:08.736838 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:08.714797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38563 (local address 127.20.121.130:58955)
0929 16:04:08.716656 (+ 1859us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:08.716709 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:08.718664 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:08.719421 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:08.719430 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:08.720009 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:08.720658 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:08.720676 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:08.722242 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:08.722245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:08.723533 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:08.723549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:08.724773 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:08.724779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:08.725036 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:08.725845 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:08.725871 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:08.728628 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:08.732377 (+ 3749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:08.732386 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:08.732400 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:08.732880 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:08.733350 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:08.733354 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:08.733357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:08.733613 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:08.734461 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:08.734472 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:08.734864 (+ 392us) client_negotiation.cc:769] Sending connection context
0929 16:04:08.735144 (+ 280us) client_negotiation.cc:240] Negotiation successful
0929 16:04:08.735596 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1541,"thread_start_us":1265,"threads_started":1}
I20260929 16:04:08.738624 21866 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38563
I20260929 16:04:08.738958 21866 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:08.739715 21866 heartbeater.cc:511] Master 127.20.121.190:38563 requested a full tablet report, sending...
I20260929 16:04:08.741580 21543 ts_manager.cc:194] Registered new tserver with Master: f2a925bc8632485f833e806a83b23a0c (127.20.121.130:45967)
I20260929 16:04:08.742635 21543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:58955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:04:08.762133 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38563
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:09.109653 21872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:09.109946 21872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:09.110108 21872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:09.118218 21872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:09.118355 21872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:09.118477 21872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:09.128873 21872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38563
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:09.130437 21872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:09.131853 21872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:09.146610 21878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:09.146487 21877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:09.146531 21880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:09.147604 21872 server_base.cc:1034] running on GCE node
I20260929 16:04:09.148447 21872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:09.149588 21872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:09.150828 21872 hybrid_clock.cc:648] HybridClock initialized: now 1790697849150779 us; error 48 us; skew 500 ppm
I20260929 16:04:09.151188 21872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:09 dist-test-slave-x07d krb5kdc[21498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697849, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:09.154721 21872 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:04:09.156473 21872 webserver.cc:460] Webserver started at http://127.20.121.131:33553/ using document root <none> and password file <none>
I20260929 16:04:09.157294 21872 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:09.157444 21872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:09.157769 21872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:09.160974 21872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "14878a1c14a24ce08a23bc57179890d5"
format_stamp: "Formatted at 2026-09-29 16:04:09 on dist-test-slave-x07d"
server_key: "0cd62c9dcfc73af767fe529082723175"
server_key_iv: "d10b2613f6a262ab818dfbcac12cad18"
server_key_version: "encryptionkey@0"
I20260929 16:04:09.161847 21872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "14878a1c14a24ce08a23bc57179890d5"
format_stamp: "Formatted at 2026-09-29 16:04:09 on dist-test-slave-x07d"
server_key: "0cd62c9dcfc73af767fe529082723175"
server_key_iv: "d10b2613f6a262ab818dfbcac12cad18"
server_key_version: "encryptionkey@0"
I20260929 16:04:09.167547 21872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260929 16:04:09.171666 21887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:09.173306 21872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:04:09.173488 21872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "14878a1c14a24ce08a23bc57179890d5"
format_stamp: "Formatted at 2026-09-29 16:04:09 on dist-test-slave-x07d"
server_key: "0cd62c9dcfc73af767fe529082723175"
server_key_iv: "d10b2613f6a262ab818dfbcac12cad18"
server_key_version: "encryptionkey@0"
I20260929 16:04:09.173686 21872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:09.202507 21872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:09.207436 21872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:09.207774 21872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:09.208679 21732 heartbeater.cc:503] Master 127.20.121.190:38563 was elected leader, sending a full tablet report...
I20260929 16:04:09.208978 21872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:09.210602 21872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:09.210706 21872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:09.210870 21872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:09.210947 21872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:09.248471 21872 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:34239
I20260929 16:04:09.248530 21999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:34239 every 8 connection(s)
I20260929 16:04:09.250296 21872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:09.251603 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 21872
I20260929 16:04:09.251799 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:09.252223 20966 external_mini_cluster.cc:1428] Setting key 26fc06b7e5ed10dd4dd478baa8581b5f
Sep 29 16:04:09 dist-test-slave-x07d krb5kdc[21498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697849, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:09.272969 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.253629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:59343 (local address 127.20.121.190:38563)
0929 16:04:09.254263 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.254268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.255452 (+ 1184us) server_negotiation.cc:407] Connection header received
0929 16:04:09.257313 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.257318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.257429 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.257543 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:09.259256 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.260033 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.261615 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.262081 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.262213 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.266000 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:09.266071 (+ 71us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:09.266079 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:09.266136 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:09.269611 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.270581 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.270585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.270588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.270765 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.271210 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.271214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.271216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.271539 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:09.271662 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.272520 (+ 858us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.272700 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":430,"threads_started":1}
I20260929 16:04:09.274566 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.254365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38563 (local address 127.20.121.131:59343)
0929 16:04:09.255293 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:09.255357 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:09.257028 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:09.257788 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:09.257798 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:09.258488 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:09.259082 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.259111 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.260198 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.260202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.261460 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.261473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.262307 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.262314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.262548 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.263242 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:09.263265 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:09.265694 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:09.269771 (+ 4077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:09.269780 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:09.269796 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:09.270462 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:09.270887 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:09.270890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:09.270893 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:09.271103 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:09.271685 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:09.271692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:09.272092 (+ 400us) client_negotiation.cc:769] Sending connection context
0929 16:04:09.272364 (+ 272us) client_negotiation.cc:240] Negotiation successful
0929 16:04:09.272824 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":300,"threads_started":1}
I20260929 16:04:09.276105 22000 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38563
I20260929 16:04:09.276435 22000 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:09.277259 22000 heartbeater.cc:511] Master 127.20.121.190:38563 requested a full tablet report, sending...
I20260929 16:04:09.278932 21543 ts_manager.cc:194] Registered new tserver with Master: 14878a1c14a24ce08a23bc57179890d5 (127.20.121.131:34239)
I20260929 16:04:09.280092 21543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:59343
I20260929 16:04:09.281517 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:04:09.302076 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.288319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.20.121.190:38563)
0929 16:04:09.288572 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.288579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.288991 (+ 412us) server_negotiation.cc:407] Connection header received
0929 16:04:09.289124 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.289128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.289217 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.289315 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:09.290927 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.291686 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.293076 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.293470 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.293582 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.296705 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:09.296710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:09.296714 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:09.296753 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:09.299451 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.300221 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.300225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.300227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.300338 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.300708 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.300711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.300713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.301019 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:09.301209 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.301521 (+ 312us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.301742 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 16:04:09.309348 21543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:04:09.317416 21543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:09.343559 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.331727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.20.121.129:38921)
0929 16:04:09.333270 (+ 1543us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.333283 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.333926 (+ 643us) server_negotiation.cc:407] Connection header received
0929 16:04:09.334070 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.334079 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.335173 (+ 1094us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.336041 (+ 868us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:09.336292 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.338130 (+ 1838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.340782 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.342355 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.342551 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.342747 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.343005 (+ 258us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.343191 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"thread_start_us":1210,"threads_started":1}
I20260929 16:04:09.343643 22014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.331244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:38921 (local address 127.0.0.1:34756)
0929 16:04:09.333738 (+ 2494us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:09.333761 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:09.333888 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:09.335383 (+ 1495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:09.335390 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:09.335419 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:09.335827 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.335841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.338247 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.338254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.340583 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.340598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.342550 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.342556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.342742 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.342808 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 16:04:09.342960 (+ 152us) client_negotiation.cc:240] Negotiation successful
0929 16:04:09.343166 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2365,"mutex_wait_us":161,"thread_start_us":2233,"threads_started":1}
I20260929 16:04:09.344327 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.331028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:45967 (local address 127.0.0.1:45202)
0929 16:04:09.332260 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:09.332319 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:09.332571 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:09.334826 (+ 2255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:09.334833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:09.334865 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:09.335253 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.335270 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.337654 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.337660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.340933 (+ 3273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.340946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.343308 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.343320 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.343530 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.343590 (+ 60us) client_negotiation.cc:769] Sending connection context
0929 16:04:09.343668 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 16:04:09.343818 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":454,"thread_start_us":655,"threads_started":1}
I20260929 16:04:09.345388 22018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.334458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54460 (local address 127.20.121.131:34239)
0929 16:04:09.335575 (+ 1117us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.335585 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.335617 (+ 32us) server_negotiation.cc:407] Connection header received
0929 16:04:09.335714 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.335723 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.335969 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.336196 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:09.337104 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.338769 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.341755 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.343479 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.343699 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.343885 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.344823 (+ 938us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.345034 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":714,"threads_started":1}
I20260929 16:04:09.345608 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.331230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45202 (local address 127.20.121.130:45967)
0929 16:04:09.333971 (+ 2741us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.333982 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.334051 (+ 69us) server_negotiation.cc:407] Connection header received
0929 16:04:09.334179 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.334192 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.334519 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.334814 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:09.335455 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.337496 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.341211 (+ 3715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.343027 (+ 1816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.344649 (+ 1622us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.344857 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.344940 (+ 83us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.345273 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2565,"thread_start_us":2450,"threads_started":1}
I20260929 16:04:09.346450 22017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.333859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:34239 (local address 127.0.0.1:54460)
0929 16:04:09.334652 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:09.334673 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:09.334801 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:09.336514 (+ 1713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:09.336520 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:09.336544 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:09.336917 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.336927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.338928 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.338933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.341573 (+ 2640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.341588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.344050 (+ 2462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.344057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.344275 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.344326 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 16:04:09.344436 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 16:04:09.346181 (+ 1745us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":371,"thread_start_us":514,"threads_started":1}
I20260929 16:04:09.348016 21667 tablet_service.cc:1467] Processing CreateTablet for tablet 7a68afaae5f24c3fa1153eca0b158699 (DEFAULT_TABLE table=test-table [id=f02905bd753b431fa2f06c45cc7695e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:09.349673 21667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a68afaae5f24c3fa1153eca0b158699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:09.350194 21801 tablet_service.cc:1467] Processing CreateTablet for tablet 7a68afaae5f24c3fa1153eca0b158699 (DEFAULT_TABLE table=test-table [id=f02905bd753b431fa2f06c45cc7695e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:09.351996 21801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a68afaae5f24c3fa1153eca0b158699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:09.353152 21935 tablet_service.cc:1467] Processing CreateTablet for tablet 7a68afaae5f24c3fa1153eca0b158699 (DEFAULT_TABLE table=test-table [id=f02905bd753b431fa2f06c45cc7695e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:09.354749 21935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a68afaae5f24c3fa1153eca0b158699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:09.363025 22019 tablet_bootstrap.cc:492] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb: Bootstrap starting.
I20260929 16:04:09.367437 22019 tablet_bootstrap.cc:654] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:09.368569 22019 log.cc:824] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:09.370395 22020 tablet_bootstrap.cc:492] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c: Bootstrap starting.
I20260929 16:04:09.372397 22022 tablet_bootstrap.cc:492] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5: Bootstrap starting.
I20260929 16:04:09.375674 22020 tablet_bootstrap.cc:654] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:09.376086 22022 tablet_bootstrap.cc:654] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:09.376871 22019 tablet_bootstrap.cc:492] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb: No bootstrap required, opened a new log
I20260929 16:04:09.377043 22020 log.cc:824] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:09.377122 22022 log.cc:824] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:09.377207 22019 ts_tablet_manager.cc:1397] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260929 16:04:09.381443 22022 tablet_bootstrap.cc:492] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5: No bootstrap required, opened a new log
I20260929 16:04:09.381748 22022 ts_tablet_manager.cc:1397] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260929 16:04:09.382314 22020 tablet_bootstrap.cc:492] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c: No bootstrap required, opened a new log
I20260929 16:04:09.382592 22020 ts_tablet_manager.cc:1397] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260929 16:04:09.389770 22019 raft_consensus.cc:348] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } }
I20260929 16:04:09.390404 22019 raft_consensus.cc:374] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:09.390590 22019 raft_consensus.cc:729] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2808e14ea0cf4a1fa134cf3f97fc10eb, State: Initialized, Role: FOLLOWER
I20260929 16:04:09.391131 22022 raft_consensus.cc:348] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } }
I20260929 16:04:09.391527 22022 raft_consensus.cc:374] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:09.391358 22019 consensus_queue.cc:260] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } }
I20260929 16:04:09.391649 22022 raft_consensus.cc:729] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14878a1c14a24ce08a23bc57179890d5, State: Initialized, Role: FOLLOWER
I20260929 16:04:09.392354 22022 consensus_queue.cc:260] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } }
I20260929 16:04:09.393586 22019 ts_tablet_manager.cc:1428] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260929 16:04:09.394657 22000 heartbeater.cc:503] Master 127.20.121.190:38563 was elected leader, sending a full tablet report...
I20260929 16:04:09.395232 22022 ts_tablet_manager.cc:1428] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5: Time spent starting tablet: real 0.013s user 0.012s sys 0.003s
I20260929 16:04:09.394418 22020 raft_consensus.cc:348] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } }
I20260929 16:04:09.396525 22020 raft_consensus.cc:374] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:09.396651 22020 raft_consensus.cc:729] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2a925bc8632485f833e806a83b23a0c, State: Initialized, Role: FOLLOWER
I20260929 16:04:09.397388 22020 consensus_queue.cc:260] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } }
I20260929 16:04:09.398756 21866 heartbeater.cc:503] Master 127.20.121.190:38563 was elected leader, sending a full tablet report...
I20260929 16:04:09.399567 22020 ts_tablet_manager.cc:1428] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
W20260929 16:04:09.423357 21733 tablet.cc:2369] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:04:09.463259 21867 tablet.cc:2369] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:04:09.490993 22026 raft_consensus.cc:482] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:09.491390 22026 raft_consensus.cc:504] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } }
I20260929 16:04:09.493350 22026 leader_election.cc:288] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2a925bc8632485f833e806a83b23a0c (127.20.121.130:45967), 2808e14ea0cf4a1fa134cf3f97fc10eb (127.20.121.129:38921)
I20260929 16:04:09.496762 22027 raft_consensus.cc:482] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:09.497205 22027 raft_consensus.cc:504] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } }
W20260929 16:04:09.502463 22001 tablet.cc:2369] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:04:09.503098 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.493715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:38921 (local address 127.20.121.131:39183)
0929 16:04:09.494044 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:09.494079 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:09.494218 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:09.494691 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:09.494696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:09.494731 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:09.494933 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.494947 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.496549 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.496556 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.498765 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.498784 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.501725 (+ 2941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.501736 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.502138 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.502204 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 16:04:09.502338 (+ 134us) client_negotiation.cc:240] Negotiation successful
0929 16:04:09.502601 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":22}
I20260929 16:04:09.504004 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.493905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:39183 (local address 127.20.121.129:38921)
0929 16:04:09.494112 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.494120 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.494143 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:04:09.494433 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.494440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.494532 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.494654 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:09.495079 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.496278 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.499007 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.501103 (+ 2096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.501346 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.501479 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.503389 (+ 1910us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.503634 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 16:04:09.504956 22027 leader_election.cc:288] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2808e14ea0cf4a1fa134cf3f97fc10eb (127.20.121.129:38921), 14878a1c14a24ce08a23bc57179890d5 (127.20.121.131:34239)
I20260929 16:04:09.505434 21687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a68afaae5f24c3fa1153eca0b158699" candidate_uuid: "14878a1c14a24ce08a23bc57179890d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" is_pre_election: true
I20260929 16:04:09.506107 21687 raft_consensus.cc:2454] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14878a1c14a24ce08a23bc57179890d5 in term 0.
I20260929 16:04:09.506985 21889 leader_election.cc:302] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14878a1c14a24ce08a23bc57179890d5, 2808e14ea0cf4a1fa134cf3f97fc10eb; no voters:
I20260929 16:04:09.507440 22026 raft_consensus.cc:2781] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:09.507606 22026 raft_consensus.cc:482] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:09.507969 22026 raft_consensus.cc:3037] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:09.508616 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.493942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:47591 (local address 127.20.121.130:45967)
0929 16:04:09.494212 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.494222 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.494814 (+ 592us) server_negotiation.cc:407] Connection header received
0929 16:04:09.494960 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.494966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.495070 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.495183 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:09.495679 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.496817 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.505792 (+ 8975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.507334 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.507555 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.507675 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.508164 (+ 489us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.508302 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":104}
I20260929 16:04:09.509526 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.493806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:45967 (local address 127.20.121.131:47591)
0929 16:04:09.494691 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:09.494713 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:09.494826 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:09.495211 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:09.495214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:09.495233 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:09.495512 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.495520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.496988 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.496993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.505547 (+ 8554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.505581 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.507739 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.507745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.507931 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.507978 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 16:04:09.508909 (+ 931us) client_negotiation.cc:240] Negotiation successful
0929 16:04:09.509088 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":26,"thread_start_us":558,"threads_started":1}
I20260929 16:04:09.511245 21821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a68afaae5f24c3fa1153eca0b158699" candidate_uuid: "14878a1c14a24ce08a23bc57179890d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a925bc8632485f833e806a83b23a0c" is_pre_election: true
I20260929 16:04:09.511969 22026 raft_consensus.cc:504] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } }
I20260929 16:04:09.512435 21821 raft_consensus.cc:2454] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14878a1c14a24ce08a23bc57179890d5 in term 0.
I20260929 16:04:09.513319 21687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a68afaae5f24c3fa1153eca0b158699" candidate_uuid: "14878a1c14a24ce08a23bc57179890d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb"
I20260929 16:04:09.513541 21687 raft_consensus.cc:3037] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:09.513896 21821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a68afaae5f24c3fa1153eca0b158699" candidate_uuid: "14878a1c14a24ce08a23bc57179890d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a925bc8632485f833e806a83b23a0c"
I20260929 16:04:09.514225 21821 raft_consensus.cc:3037] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:09.516907 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.499527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:49181 (local address 127.20.121.129:38921)
0929 16:04:09.504594 (+ 5067us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.504605 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.505527 (+ 922us) server_negotiation.cc:407] Connection header received
0929 16:04:09.505896 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.505906 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.506031 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.506131 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:09.508173 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.509834 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.513898 (+ 4064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.515420 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.515608 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.515699 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.516366 (+ 667us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.516533 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4909,"thread_start_us":5831,"threads_started":1}
I20260929 16:04:09.517185 21687 raft_consensus.cc:2454] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14878a1c14a24ce08a23bc57179890d5 in term 1.
I20260929 16:04:09.517699 21889 leader_election.cc:302] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14878a1c14a24ce08a23bc57179890d5, 2808e14ea0cf4a1fa134cf3f97fc10eb; no voters:
I20260929 16:04:09.517808 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.499338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:38921 (local address 127.20.121.130:49181)
0929 16:04:09.505353 (+ 6015us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:09.505382 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:09.505694 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:09.506456 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:09.506465 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:09.506503 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:09.507948 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.507968 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.510278 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.510283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.513678 (+ 3395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.513694 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.515926 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.515932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.516146 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.516195 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 16:04:09.517170 (+ 975us) client_negotiation.cc:240] Negotiation successful
0929 16:04:09.517327 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5769,"mutex_wait_us":41,"thread_start_us":5466,"threads_started":1}
I20260929 16:04:09.517750 21821 raft_consensus.cc:2454] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14878a1c14a24ce08a23bc57179890d5 in term 1.
I20260929 16:04:09.518421 22032 raft_consensus.cc:2781] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:09.518788 21686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a68afaae5f24c3fa1153eca0b158699" candidate_uuid: "f2a925bc8632485f833e806a83b23a0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" is_pre_election: true
I20260929 16:04:09.519439 22018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.499924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:44093 (local address 127.20.121.131:34239)
0929 16:04:09.500669 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.500678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.510530 (+ 9852us) server_negotiation.cc:407] Connection header received
0929 16:04:09.510780 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.510785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.510854 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.510909 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:09.512453 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.513506 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.516293 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.517608 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.517743 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.517816 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.518529 (+ 713us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.518691 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260929 16:04:09.519582 22032 raft_consensus.cc:686] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [term 1 LEADER]: Becoming Leader. State: Replica: 14878a1c14a24ce08a23bc57179890d5, State: Running, Role: LEADER
I20260929 16:04:09.519696 21686 raft_consensus.cc:2379] T 7a68afaae5f24c3fa1153eca0b158699 P 2808e14ea0cf4a1fa134cf3f97fc10eb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2a925bc8632485f833e806a83b23a0c in current term 1: Already voted for candidate 14878a1c14a24ce08a23bc57179890d5 in this term.
I20260929 16:04:09.520222 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.499426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:34239 (local address 127.20.121.130:44093)
0929 16:04:09.510415 (+ 10989us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:09.510444 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:09.510637 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:09.511084 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:09.511093 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:09.511117 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:09.512253 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.512266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.513704 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.513709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.516084 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:09.516100 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.518049 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.518068 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:09.518280 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.518349 (+ 69us) client_negotiation.cc:769] Sending connection context
0929 16:04:09.519606 (+ 1257us) client_negotiation.cc:240] Negotiation successful
0929 16:04:09.519798 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10829,"mutex_wait_us":1,"thread_start_us":10625,"threads_started":1}
I20260929 16:04:09.520191 22032 consensus_queue.cc:237] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } }
I20260929 16:04:09.521359 22026 leader_election.cc:288] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [CANDIDATE]: Term 1 election: Requested vote from peers f2a925bc8632485f833e806a83b23a0c (127.20.121.130:45967), 2808e14ea0cf4a1fa134cf3f97fc10eb (127.20.121.129:38921)
I20260929 16:04:09.521638 21955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a68afaae5f24c3fa1153eca0b158699" candidate_uuid: "f2a925bc8632485f833e806a83b23a0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14878a1c14a24ce08a23bc57179890d5" is_pre_election: true
I20260929 16:04:09.522684 21757 leader_election.cc:302] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2a925bc8632485f833e806a83b23a0c; no voters: 14878a1c14a24ce08a23bc57179890d5, 2808e14ea0cf4a1fa134cf3f97fc10eb
I20260929 16:04:09.524446 22027 raft_consensus.cc:2726] T 7a68afaae5f24c3fa1153eca0b158699 P f2a925bc8632485f833e806a83b23a0c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 16:04:09.525542 21541 catalog_manager.cc:5495] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14878a1c14a24ce08a23bc57179890d5 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "14878a1c14a24ce08a23bc57179890d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2808e14ea0cf4a1fa134cf3f97fc10eb" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14878a1c14a24ce08a23bc57179890d5" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34239 } health_report { overall_health: HEALTHY } } }
Sep 29 16:04:09 dist-test-slave-x07d krb5kdc[21498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:09.594396 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.576629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.20.121.129:38921)
0929 16:04:09.576863 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.576872 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.577267 (+ 395us) server_negotiation.cc:407] Connection header received
0929 16:04:09.577519 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.577523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.577614 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.577726 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:09.579457 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.580259 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.581616 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.582155 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.582295 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.586871 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:09.586889 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:09.586905 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:09.586949 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:09.590919 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.591870 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.591879 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.591882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.592067 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.592537 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.592545 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.592549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.592968 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:09.593353 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.593693 (+ 340us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.593971 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
Sep 29 16:04:09 dist-test-slave-x07d krb5kdc[21498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697849, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 16:04:09 dist-test-slave-x07d krb5kdc[21498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697849, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:09.645519 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.630302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34776 (local address 127.20.121.129:38921)
0929 16:04:09.630504 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.630513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.630861 (+ 348us) server_negotiation.cc:407] Connection header received
0929 16:04:09.631088 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.631093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.631161 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.631268 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:09.632726 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.633504 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.634949 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.635383 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.635532 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.639673 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:09.639679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:09.639683 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:09.639724 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:09.642742 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.643504 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.643509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.643511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.643644 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.644089 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.644094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.644096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.644377 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:09.644641 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.644924 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.645143 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Sep 29 16:04:09 dist-test-slave-x07d krb5kdc[21498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 16:04:09 dist-test-slave-x07d krb5kdc[21498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:09.692176 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.677030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34786 (local address 127.20.121.129:38921)
0929 16:04:09.677224 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.677234 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.677618 (+ 384us) server_negotiation.cc:407] Connection header received
0929 16:04:09.677829 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.677834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.677898 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.678009 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:09.679477 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.680238 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.681532 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.681958 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.682159 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.686600 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:09.686606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:09.686609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:09.686650 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:09.689417 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.690240 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.690246 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.690254 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.690402 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.690786 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.690790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.690792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.691069 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:09.691330 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.691634 (+ 304us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.691848 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260929 16:04:09.693984 21647 tablet_service.cc:2908] Rejecting scan request for tablet 7a68afaae5f24c3fa1153eca0b158699: Uninitialized: clean time has not yet been initialized
I20260929 16:04:09.697799 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 21604
I20260929 16:04:09.712236 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.698876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.20.121.129:38921)
0929 16:04:09.699025 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.699029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.699541 (+ 512us) server_negotiation.cc:407] Connection header received
0929 16:04:09.699688 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.699693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.699762 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.699862 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:09.701185 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.701900 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.703224 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.703642 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.703745 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.706317 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:09.706323 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:09.706327 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:09.706363 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:09.709320 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.710147 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.710155 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.710159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.710297 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.710706 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.710715 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.710719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.711023 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:09.711308 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.711626 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.711869 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 16:04:09.713035 21727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:09.810127 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 21604
I20260929 16:04:09.828862 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 21738
Sep 29 16:04:09 dist-test-slave-x07d krb5kdc[21498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:04:09.845198 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.829800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45216 (local address 127.20.121.130:45967)
0929 16:04:09.830122 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.830127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.830145 (+ 18us) server_negotiation.cc:407] Connection header received
0929 16:04:09.830203 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.830207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.830276 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.830360 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:09.831609 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.832402 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.833701 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.834203 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.834300 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.838486 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:09.838502 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:09.838519 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:09.838580 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:09.841602 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.842684 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.842693 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.842695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.842873 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.843366 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.843370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.843373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.843845 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:09.844212 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.844520 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.844720 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260929 16:04:09.845988 21861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:09.940389 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 21738
I20260929 16:04:09.941248 22032 consensus_queue.cc:1035] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2a925bc8632485f833e806a83b23a0c" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 16:04:09.958061 21890 connection.cc:535] client connection to 127.20.121.130:45967 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 16:04:09.958371 21890 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 16:04:09.958750 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 21872
I20260929 16:04:09.960106 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.959682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:45967 (local address 127.20.121.131:35295)
0929 16:04:09.959824 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:09.959945 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.130:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 16:04:09.960450 21890 consensus_peers.cc:487] T 7a68afaae5f24c3fa1153eca0b158699 P 14878a1c14a24ce08a23bc57179890d5 -> Peer f2a925bc8632485f833e806a83b23a0c (127.20.121.130:45967): Couldn't send request to peer f2a925bc8632485f833e806a83b23a0c. Status: Network error: Client connection negotiation failed: client connection to 127.20.121.130:45967: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Sep 29 16:04:09 dist-test-slave-x07d krb5kdc[21498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:04:09.975980 22018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:09.959616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54464 (local address 127.20.121.131:34239)
0929 16:04:09.959852 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:09.959857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:09.959878 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:04:09.959940 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:09.959944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:09.960009 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:09.960173 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:09.961529 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.962681 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.964279 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:09.964758 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:09.964870 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:09.969624 (+ 4754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:09.969645 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:09.969662 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:09.969709 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:09.972608 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.973527 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.973534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.973536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.973679 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:09.974386 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:09.974389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:09.974392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:09.974787 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:09.975021 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:09.975410 (+ 389us) server_negotiation.cc:299] Negotiation successful
0929 16:04:09.975582 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260929 16:04:09.976766 21995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:10.070479 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 21872
I20260929 16:04:10.087513 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 21511
I20260929 16:04:10.088979 21573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:10.203429 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 21511
2026-09-29T16:04:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4448 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:10 dist-test-slave-x07d krb5kdc[22053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:10 dist-test-slave-x07d krb5kdc[22053](info): set up 2 sockets
Sep 29 16:04:10 dist-test-slave-x07d krb5kdc[22053](info): commencing operation
krb5kdc: starting...
W20260929 16:04:11.350397 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:11 dist-test-slave-x07d krb5kdc[22053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:04:11.547137 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:40505
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:40505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:11.889680 22066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:11.889992 22066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:11.890185 22066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:11.897643 22066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:11.897773 22066 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 16:04:11.897850 22066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:11.897996 22066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:11.898166 22066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:11.898272 22066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:11.908831 22066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:40505
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:40505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:11.910250 22066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:11.911625 22066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:11.925067 22071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:11.925038 22074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:11.925189 22072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:11.927825 22066 server_base.cc:1034] running on GCE node
I20260929 16:04:11.928807 22066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:11.930413 22066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:11.931671 22066 hybrid_clock.cc:648] HybridClock initialized: now 1790697851931573 us; error 55 us; skew 500 ppm
I20260929 16:04:11.931993 22066 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:11 dist-test-slave-x07d krb5kdc[22053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697851, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:11.935601 22066 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:04:11.937237 22066 webserver.cc:460] Webserver started at http://127.20.121.190:38525/ using document root <none> and password file <none>
I20260929 16:04:11.937987 22066 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:11.938166 22066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:11.938498 22066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:11.941601 22066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "9886447215454108a7b672135c64d737"
format_stamp: "Formatted at 2026-09-29 16:04:11 on dist-test-slave-x07d"
server_key: "da49c6d48649aa6ee3cb07db15f22a13"
server_key_iv: "996769fb0f5954a3ba39ce0b68ddb3fc"
server_key_version: "encryptionkey@0"
I20260929 16:04:11.942463 22066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "9886447215454108a7b672135c64d737"
format_stamp: "Formatted at 2026-09-29 16:04:11 on dist-test-slave-x07d"
server_key: "da49c6d48649aa6ee3cb07db15f22a13"
server_key_iv: "996769fb0f5954a3ba39ce0b68ddb3fc"
server_key_version: "encryptionkey@0"
I20260929 16:04:11.948308 22066 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:04:11.952345 22081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:11.953909 22066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:04:11.954160 22066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "9886447215454108a7b672135c64d737"
format_stamp: "Formatted at 2026-09-29 16:04:11 on dist-test-slave-x07d"
server_key: "da49c6d48649aa6ee3cb07db15f22a13"
server_key_iv: "996769fb0f5954a3ba39ce0b68ddb3fc"
server_key_version: "encryptionkey@0"
I20260929 16:04:11.954399 22066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:11.968626 22066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:11.972792 22066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:11.973076 22066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:11.996820 22066 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:40505
I20260929 16:04:11.996840 22142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:40505 every 8 connection(s)
I20260929 16:04:11.998492 22066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:12.002512 22143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:12.007745 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 22066
I20260929 16:04:12.008056 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:12.008697 20966 external_mini_cluster.cc:1428] Setting key f063ecfeac638044c9e12df13fd80039
I20260929 16:04:12.014410 22143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737: Bootstrap starting.
I20260929 16:04:12.018293 22143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:12.019474 22143 log.cc:824] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:12.022779 22143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737: No bootstrap required, opened a new log
Sep 29 16:04:12 dist-test-slave-x07d krb5kdc[22053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:12.033262 22143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9886447215454108a7b672135c64d737" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40505 } }
I20260929 16:04:12.033742 22143 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:12.033859 22143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9886447215454108a7b672135c64d737, State: Initialized, Role: FOLLOWER
I20260929 16:04:12.034627 22143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9886447215454108a7b672135c64d737" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40505 } }
I20260929 16:04:12.034915 22143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:12.034958 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:12.010971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56952 (local address 127.20.121.190:40505)
0929 16:04:12.011941 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:12.011957 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:12.012008 (+ 51us) server_negotiation.cc:407] Connection header received
0929 16:04:12.014467 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:12.014501 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:12.015084 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:12.015552 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:12.016828 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:12.018859 (+ 2031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:12.020333 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:12.021242 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:12.021428 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:12.025648 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:12.025660 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:12.025683 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:12.025735 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:12.029896 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:12.030927 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:12.030937 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:12.030948 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:12.031189 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:12.031534 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:12.031540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:12.031543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:12.032315 (+ 772us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:12.032596 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:12.033001 (+ 405us) server_negotiation.cc:299] Negotiation successful
0929 16:04:12.033447 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":455,"threads_started":1}
I20260929 16:04:12.035092 22143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:12.035331 22143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:12.037701 22143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9886447215454108a7b672135c64d737" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40505 } }
I20260929 16:04:12.038272 22143 leader_election.cc:302] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9886447215454108a7b672135c64d737; no voters:
I20260929 16:04:12.038792 22143 leader_election.cc:288] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:12.038977 22148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:12.039943 22148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [term 1 LEADER]: Becoming Leader. State: Replica: 9886447215454108a7b672135c64d737, State: Running, Role: LEADER
I20260929 16:04:12.040580 22148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9886447215454108a7b672135c64d737" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40505 } }
I20260929 16:04:12.042101 22143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:12.043483 22149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9886447215454108a7b672135c64d737. Latest consensus state: current_term: 1 leader_uuid: "9886447215454108a7b672135c64d737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9886447215454108a7b672135c64d737" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40505 } } }
I20260929 16:04:12.043545 22150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9886447215454108a7b672135c64d737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9886447215454108a7b672135c64d737" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40505 } } }
I20260929 16:04:12.043952 22150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:12.046679 22149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:12.047683 22153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:12.052991 22153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:12.063969 22153 catalog_manager.cc:1347] Generated new cluster ID: 9b6276d85a0944059b7ef8e3573fc708
I20260929 16:04:12.064172 22153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:12.091009 22153 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:12.093436 22153 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:12.102761 22153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737: Generated new TSK 0
I20260929 16:04:12.104046 22153 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:04:12.171308 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:40505
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:12.514858 22169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:12.515226 22169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:12.515396 22169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:12.523106 22169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:12.523283 22169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:12.523429 22169 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 16:04:12.523567 22169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:12.533809 22169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.121.190:40505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:12.535209 22169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:12.536585 22169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:12.550611 22174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:12.550725 22175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:12.550688 22177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:12.551898 22169 server_base.cc:1034] running on GCE node
I20260929 16:04:12.552649 22169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:12.553643 22169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:12.554895 22169 hybrid_clock.cc:648] HybridClock initialized: now 1790697852554791 us; error 84 us; skew 500 ppm
I20260929 16:04:12.555351 22169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:12 dist-test-slave-x07d krb5kdc[22053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697852, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:12.571522 22169 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:04:12.573262 22169 webserver.cc:460] Webserver started at http://127.20.121.129:45839/ using document root <none> and password file <none>
I20260929 16:04:12.574128 22169 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:12.574262 22169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:12.574594 22169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:12.577709 22169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "0ca2699bb9334e37b8118616cce5b454"
format_stamp: "Formatted at 2026-09-29 16:04:12 on dist-test-slave-x07d"
server_key: "0d482250fcde72e31632c2b9be029e5d"
server_key_iv: "12b93f0c38dfbf31129061b8ef47fe71"
server_key_version: "encryptionkey@0"
I20260929 16:04:12.578657 22169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "0ca2699bb9334e37b8118616cce5b454"
format_stamp: "Formatted at 2026-09-29 16:04:12 on dist-test-slave-x07d"
server_key: "0d482250fcde72e31632c2b9be029e5d"
server_key_iv: "12b93f0c38dfbf31129061b8ef47fe71"
server_key_version: "encryptionkey@0"
I20260929 16:04:12.584517 22169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:04:12.588960 22184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:12.590755 22169 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:04:12.591078 22169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "0ca2699bb9334e37b8118616cce5b454"
format_stamp: "Formatted at 2026-09-29 16:04:12 on dist-test-slave-x07d"
server_key: "0d482250fcde72e31632c2b9be029e5d"
server_key_iv: "12b93f0c38dfbf31129061b8ef47fe71"
server_key_version: "encryptionkey@0"
I20260929 16:04:12.591342 22169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:12.612661 22169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:12.618295 22169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:12.618606 22169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:12.621863 22169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:12.621991 22169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:12.622184 22169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:12.622267 22169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:04:12 dist-test-slave-x07d krb5kdc[22053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697852, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:12.654573 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:12.623043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:56377 (local address 127.20.121.190:40505)
0929 16:04:12.623652 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:12.623660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:12.625192 (+ 1532us) server_negotiation.cc:407] Connection header received
0929 16:04:12.629117 (+ 3925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:12.629125 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:12.629239 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:12.629389 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:12.633083 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:12.634613 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:12.638378 (+ 3765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:12.638997 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:12.639169 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:12.646347 (+ 7178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:12.646356 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:12.646360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:12.646409 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:12.650104 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:12.651096 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:12.651104 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:12.651108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:12.651250 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:12.652014 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:12.652019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:12.652023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:12.652461 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:12.652625 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:12.653860 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0929 16:04:12.654176 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":260,"threads_started":1}
I20260929 16:04:12.656217 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:12.623806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40505 (local address 127.20.121.129:56377)
0929 16:04:12.624908 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:12.624995 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:12.628656 (+ 3661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:12.629979 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:12.629998 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:12.631254 (+ 1256us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:12.632696 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:12.632741 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:12.634934 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:12.634939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:12.638136 (+ 3197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:12.638161 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:12.639332 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:12.639344 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:12.639827 (+ 483us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:12.640939 (+ 1112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:12.640992 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:12.645882 (+ 4890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:12.650309 (+ 4427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:12.650327 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:12.650345 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:12.650915 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:12.651384 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:12.651389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:12.651397 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:12.651785 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:12.652595 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:12.652609 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:12.653299 (+ 690us) client_negotiation.cc:769] Sending connection context
0929 16:04:12.653732 (+ 433us) client_negotiation.cc:240] Negotiation successful
0929 16:04:12.654405 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":363,"threads_started":1}
I20260929 16:04:12.672107 22169 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:40013
I20260929 16:04:12.672209 22304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:40013 every 8 connection(s)
I20260929 16:04:12.673789 22169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:12.676064 22305 heartbeater.cc:347] Connected to a master server at 127.20.121.190:40505
I20260929 16:04:12.676319 22305 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:12.677152 22305 heartbeater.cc:511] Master 127.20.121.190:40505 requested a full tablet report, sending...
I20260929 16:04:12.679613 22098 ts_manager.cc:194] Registered new tserver with Master: 0ca2699bb9334e37b8118616cce5b454 (127.20.121.129:40013)
I20260929 16:04:12.681111 22098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:56377
I20260929 16:04:12.683004 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 22169
I20260929 16:04:12.683243 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:12.683996 20966 external_mini_cluster.cc:1428] Setting key 2762087ad6f458c93c18e8939428b477
I20260929 16:04:12.685491 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:12.670943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:50049 (local address 127.20.121.190:40505)
0929 16:04:12.671161 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:12.671170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:12.672327 (+ 1157us) server_negotiation.cc:407] Connection header received
0929 16:04:12.672558 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:12.672565 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:12.672658 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:12.672781 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:12.673709 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:12.674915 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:12.676498 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:12.677126 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:12.677262 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:12.679445 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:12.679451 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:12.679455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:12.679487 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:12.682409 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:12.683318 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:12.683323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:12.683327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:12.683489 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:12.683915 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:12.683922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:12.683925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:12.684241 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:12.684419 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:12.684852 (+ 433us) server_negotiation.cc:299] Negotiation successful
0929 16:04:12.685084 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 16:04:12.685503 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:12.670909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40505 (local address 127.20.121.129:50049)
0929 16:04:12.672165 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:12.672191 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:12.672394 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:12.672813 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:12.672817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:12.673244 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:12.673541 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:12.673550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:12.675062 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:12.675069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:12.676341 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:12.676353 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:12.677352 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:12.677356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:12.677486 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:12.678175 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:12.678192 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:12.679186 (+ 994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:12.682548 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:12.682554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:12.682562 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:12.683190 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:12.683617 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:12.683624 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:12.683628 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:12.683791 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:12.684370 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:12.684375 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:12.684577 (+ 202us) client_negotiation.cc:769] Sending connection context
0929 16:04:12.684753 (+ 176us) client_negotiation.cc:240] Negotiation successful
0929 16:04:12.685075 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"thread_start_us":933,"threads_started":1}
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:04:12.731138 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:40505
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:13.079535 22309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:13.079914 22309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:13.080089 22309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:13.087949 22309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:13.088147 22309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:13.088244 22309 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 16:04:13.088328 22309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:13.098693 22309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.121.190:40505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:13.100064 22309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:13.101451 22309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:13.115875 22314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:13.115818 22317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:13.115860 22315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:13.116990 22309 server_base.cc:1034] running on GCE node
I20260929 16:04:13.117597 22309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:13.118736 22309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:13.119967 22309 hybrid_clock.cc:648] HybridClock initialized: now 1790697853119899 us; error 42 us; skew 500 ppm
I20260929 16:04:13.120456 22309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:13 dist-test-slave-x07d krb5kdc[22053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697853, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:13.123986 22309 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:04:13.125627 22309 webserver.cc:460] Webserver started at http://127.20.121.130:44271/ using document root <none> and password file <none>
I20260929 16:04:13.126451 22309 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:13.126547 22309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:13.126827 22309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:13.129882 22309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "c937ec9283234120b42d86d3c7fe5161"
format_stamp: "Formatted at 2026-09-29 16:04:13 on dist-test-slave-x07d"
server_key: "dfe5b659fcf049eb3a6d69dc0a222353"
server_key_iv: "45645b626e2f211c3e37db1e5e886103"
server_key_version: "encryptionkey@0"
I20260929 16:04:13.130863 22309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "c937ec9283234120b42d86d3c7fe5161"
format_stamp: "Formatted at 2026-09-29 16:04:13 on dist-test-slave-x07d"
server_key: "dfe5b659fcf049eb3a6d69dc0a222353"
server_key_iv: "45645b626e2f211c3e37db1e5e886103"
server_key_version: "encryptionkey@0"
I20260929 16:04:13.136598 22309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:04:13.140836 22324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:13.142789 22309 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:04:13.143002 22309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "c937ec9283234120b42d86d3c7fe5161"
format_stamp: "Formatted at 2026-09-29 16:04:13 on dist-test-slave-x07d"
server_key: "dfe5b659fcf049eb3a6d69dc0a222353"
server_key_iv: "45645b626e2f211c3e37db1e5e886103"
server_key_version: "encryptionkey@0"
I20260929 16:04:13.143247 22309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:13.166442 22309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:13.171536 22309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:13.171845 22309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:13.174719 22309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:13.174849 22309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:13.175006 22309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:13.175112 22309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:04:13 dist-test-slave-x07d krb5kdc[22053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697853, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:13.205325 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.176542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40505 (local address 127.20.121.130:47829)
0929 16:04:13.177595 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:13.177660 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:13.180221 (+ 2561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:13.181657 (+ 1436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:13.181676 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:13.182738 (+ 1062us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:13.183875 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.183899 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.185109 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.185115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.187335 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.187362 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.188690 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.188703 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.189130 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.190288 (+ 1158us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:13.190342 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:13.194204 (+ 3862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:13.198653 (+ 4449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:13.198671 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:13.198698 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:13.199619 (+ 921us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:13.200264 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:13.200273 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:13.200284 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:13.200707 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:13.201557 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:13.201574 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:13.202225 (+ 651us) client_negotiation.cc:769] Sending connection context
0929 16:04:13.202693 (+ 468us) client_negotiation.cc:240] Negotiation successful
0929 16:04:13.203492 (+ 799us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":455,"threads_started":1}
I20260929 16:04:13.205868 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.175688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:47829 (local address 127.20.121.190:40505)
0929 16:04:13.175896 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:13.175903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:13.177842 (+ 1939us) server_negotiation.cc:407] Connection header received
0929 16:04:13.180727 (+ 2885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:13.180733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:13.180829 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:13.180925 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:13.184106 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.184922 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.187615 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.188360 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.188555 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.194536 (+ 5981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:13.194542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:13.194546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:13.194588 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:13.198378 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:13.199871 (+ 1493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:13.199878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:13.199882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:13.200115 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:13.200879 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:13.200886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:13.200889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:13.201350 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:13.201543 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:13.205385 (+ 3842us) server_negotiation.cc:299] Negotiation successful
0929 16:04:13.205601 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:04:13.229389 22309 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:35529
I20260929 16:04:13.229449 22443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:35529 every 8 connection(s)
I20260929 16:04:13.230970 22309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:13.232548 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 22309
I20260929 16:04:13.232752 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:13.233255 22444 heartbeater.cc:347] Connected to a master server at 127.20.121.190:40505
I20260929 16:04:13.233309 20966 external_mini_cluster.cc:1428] Setting key f5cf9c73d6da63c1104743f620080979
I20260929 16:04:13.233520 22444 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:13.235723 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.214346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:37285 (local address 127.20.121.190:40505)
0929 16:04:13.214614 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:13.214624 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:13.218193 (+ 3569us) server_negotiation.cc:407] Connection header received
0929 16:04:13.218595 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:13.218604 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:13.218708 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:13.218842 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:13.220395 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.221645 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.224646 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.225425 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.225586 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.228437 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:13.228446 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:13.228453 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:13.228508 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:13.231742 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:13.232915 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:13.232925 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:13.232931 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:13.233181 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:13.233726 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:13.233735 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:13.233741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:13.234354 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:13.234489 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:13.235096 (+ 607us) server_negotiation.cc:299] Negotiation successful
0929 16:04:13.235375 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 16:04:13.235493 22444 heartbeater.cc:511] Master 127.20.121.190:40505 requested a full tablet report, sending...
I20260929 16:04:13.235888 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.213863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40505 (local address 127.20.121.130:37285)
0929 16:04:13.217979 (+ 4116us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:13.218040 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:13.218357 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:13.218899 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:13.218904 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:13.219726 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:13.220189 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.220207 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.221829 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.221838 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.224325 (+ 2487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.224349 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.225690 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.225696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.225928 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.226802 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:13.226825 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:13.228094 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:13.231925 (+ 3831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:13.231934 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:13.231942 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:13.232719 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:13.233373 (+ 654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:13.233380 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:13.233387 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:13.233575 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:13.234575 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:13.234579 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:13.234789 (+ 210us) client_negotiation.cc:769] Sending connection context
0929 16:04:13.234937 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 16:04:13.235451 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3870,"thread_start_us":3602,"threads_started":1}
I20260929 16:04:13.237927 22098 ts_manager.cc:194] Registered new tserver with Master: c937ec9283234120b42d86d3c7fe5161 (127.20.121.130:35529)
I20260929 16:04:13.239228 22098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:47829
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:04:13.278230 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:40505
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:35811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:13.614601 22448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:13.614900 22448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:13.615087 22448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:13.622938 22448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:13.623070 22448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:13.623208 22448 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 16:04:13.623317 22448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:13.633436 22448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.121.190:40505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:13.634950 22448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:13.636484 22448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:13.651192 22453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:13.651238 22456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:13.651167 22454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:13.652372 22448 server_base.cc:1034] running on GCE node
I20260929 16:04:13.653059 22448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:13.654203 22448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:13.655458 22448 hybrid_clock.cc:648] HybridClock initialized: now 1790697853655334 us; error 95 us; skew 500 ppm
I20260929 16:04:13.655803 22448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:13 dist-test-slave-x07d krb5kdc[22053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697853, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:13.659395 22448 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:04:13.661250 22448 webserver.cc:460] Webserver started at http://127.20.121.131:46459/ using document root <none> and password file <none>
I20260929 16:04:13.662107 22448 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:13.662215 22448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:13.662504 22448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:13.665690 22448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "9b5b235ed82f48a58a56341d93205261"
format_stamp: "Formatted at 2026-09-29 16:04:13 on dist-test-slave-x07d"
server_key: "443e024c83644420d58c44560d39a02c"
server_key_iv: "4242e112fd8151890ef7fdd8dce26ad4"
server_key_version: "encryptionkey@0"
I20260929 16:04:13.666579 22448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "9b5b235ed82f48a58a56341d93205261"
format_stamp: "Formatted at 2026-09-29 16:04:13 on dist-test-slave-x07d"
server_key: "443e024c83644420d58c44560d39a02c"
server_key_iv: "4242e112fd8151890ef7fdd8dce26ad4"
server_key_version: "encryptionkey@0"
I20260929 16:04:13.672333 22448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:04:13.676174 22463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:13.677651 22448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:04:13.677815 22448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "9b5b235ed82f48a58a56341d93205261"
format_stamp: "Formatted at 2026-09-29 16:04:13 on dist-test-slave-x07d"
server_key: "443e024c83644420d58c44560d39a02c"
server_key_iv: "4242e112fd8151890ef7fdd8dce26ad4"
server_key_version: "encryptionkey@0"
I20260929 16:04:13.678081 22448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:13.684924 22305 heartbeater.cc:503] Master 127.20.121.190:40505 was elected leader, sending a full tablet report...
I20260929 16:04:13.707913 22448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:13.712715 22448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:13.713024 22448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:13.716100 22448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:13.716214 22448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:13.716346 22448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:13.716425 22448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:04:13 dist-test-slave-x07d krb5kdc[22053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697853, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:13.746143 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.716885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:39057 (local address 127.20.121.190:40505)
0929 16:04:13.717118 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:13.717125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:13.718811 (+ 1686us) server_negotiation.cc:407] Connection header received
0929 16:04:13.721641 (+ 2830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:13.721652 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:13.721774 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:13.721905 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:13.724838 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.726003 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.729567 (+ 3564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.730277 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.730453 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.737219 (+ 6766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:13.737229 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:13.737237 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:13.737287 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:13.741468 (+ 4181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:13.742848 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:13.742879 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:13.742886 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:13.743111 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:13.743684 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:13.743693 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:13.743699 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:13.744183 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:13.744360 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:13.745200 (+ 840us) server_negotiation.cc:299] Negotiation successful
0929 16:04:13.745551 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 16:04:13.747417 22472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.717648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40505 (local address 127.20.121.131:39057)
0929 16:04:13.718618 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:13.718670 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:13.721110 (+ 2440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:13.722309 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:13.722330 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:13.723358 (+ 1028us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:13.724599 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.724639 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.726229 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.726239 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.729319 (+ 3080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.729347 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.730632 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.730644 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.731092 (+ 448us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.732214 (+ 1122us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:13.732250 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:13.736767 (+ 4517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:13.741755 (+ 4988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:13.741769 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:13.741797 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:13.742662 (+ 865us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:13.743299 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:13.743303 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:13.743306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:13.743547 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:13.744348 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:13.744355 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:13.744762 (+ 407us) client_negotiation.cc:769] Sending connection context
0929 16:04:13.745010 (+ 248us) client_negotiation.cc:240] Negotiation successful
0929 16:04:13.745505 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":444,"threads_started":1}
I20260929 16:04:13.771034 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.754821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:46291 (local address 127.20.121.190:40505)
0929 16:04:13.755087 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:13.755093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:13.755871 (+ 778us) server_negotiation.cc:407] Connection header received
0929 16:04:13.756088 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:13.756095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:13.756201 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:13.756364 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:13.757650 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.759073 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.760721 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.761630 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.761808 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.763932 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:13.763945 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:13.763949 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:13.763984 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:13.767986 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:13.768976 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:13.768984 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:13.768987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:13.769134 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:13.769500 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:13.769505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:13.769509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:13.769789 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:13.769942 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:13.770417 (+ 475us) server_negotiation.cc:299] Negotiation successful
0929 16:04:13.770648 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 16:04:13.771134 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.754934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40505 (local address 127.20.121.131:46291)
0929 16:04:13.755746 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:13.755763 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:13.755936 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:13.756398 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:13.756403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:13.757075 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:13.757474 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.757485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.759254 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.759260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.760568 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.760587 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.761837 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.761842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.761967 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.762799 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:13.762831 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:13.763723 (+ 892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:13.768193 (+ 4470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:13.768197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:13.768200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:13.768830 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:13.769250 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:13.769255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:13.769259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:13.769382 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:13.769904 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:13.769909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:13.770161 (+ 252us) client_negotiation.cc:769] Sending connection context
0929 16:04:13.770326 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 16:04:13.770676 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":425,"threads_started":1}
I20260929 16:04:13.776743 22448 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:37907
I20260929 16:04:13.776767 22582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:37907 every 8 connection(s)
I20260929 16:04:13.778146 22448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:13.780181 22583 heartbeater.cc:347] Connected to a master server at 127.20.121.190:40505
I20260929 16:04:13.780422 22583 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:13.781059 22583 heartbeater.cc:511] Master 127.20.121.190:40505 requested a full tablet report, sending...
I20260929 16:04:13.782472 22098 ts_manager.cc:194] Registered new tserver with Master: 9b5b235ed82f48a58a56341d93205261 (127.20.121.131:37907)
I20260929 16:04:13.783377 22098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:39057
I20260929 16:04:13.788048 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 22448
I20260929 16:04:13.788228 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:13.788647 20966 external_mini_cluster.cc:1428] Setting key 6e142866a94e6e0affa66e7c27138a06
I20260929 16:04:13.791816 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 16:04:13 dist-test-slave-x07d krb5kdc[22053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697853, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 16:04:13 dist-test-slave-x07d krb5kdc[22053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697853, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:13.839308 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.824403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56964 (local address 127.20.121.190:40505)
0929 16:04:13.824643 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:13.824650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:13.824864 (+ 214us) server_negotiation.cc:407] Connection header received
0929 16:04:13.825104 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:13.825109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:13.825177 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:13.825289 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:13.826884 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.827730 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.829163 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.829647 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.829802 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.833804 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:13.833811 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:13.833815 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:13.833859 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:13.836689 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:13.837361 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:13.837366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:13.837368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:13.837526 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:13.837853 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:13.837857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:13.837859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:13.838282 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:13.838476 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:13.838752 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 16:04:13.838944 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 16:04:13.843808 22098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:04:13.851125 22098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:13.874449 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.864234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.20.121.129:40013)
0929 16:04:13.865958 (+ 1724us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:13.865970 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:13.866003 (+ 33us) server_negotiation.cc:407] Connection header received
0929 16:04:13.866152 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:13.866161 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:13.866426 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:13.866632 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:13.867394 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.868718 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.871914 (+ 3196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.873212 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.873394 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.873588 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:13.873942 (+ 354us) server_negotiation.cc:299] Negotiation successful
0929 16:04:13.874148 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1578,"thread_start_us":1448,"threads_started":1}
I20260929 16:04:13.874686 22597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.864010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40013 (local address 127.0.0.1:55744)
0929 16:04:13.865247 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:13.865282 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:13.865441 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:13.866626 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:13.866632 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:13.866657 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:13.867032 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.867048 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.868844 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.868850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.871723 (+ 2873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.871739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.873447 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.873453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.873651 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.873712 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 16:04:13.873879 (+ 167us) client_negotiation.cc:240] Negotiation successful
0929 16:04:13.874141 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"mutex_wait_us":34,"thread_start_us":2163,"threads_started":1}
I20260929 16:04:13.875204 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.864743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.20.121.130:35529)
0929 16:04:13.866294 (+ 1551us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:13.866302 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:13.866404 (+ 102us) server_negotiation.cc:407] Connection header received
0929 16:04:13.866607 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:13.866614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:13.866860 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:13.867046 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:13.867976 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.869503 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.872098 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.873324 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.873491 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.873673 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:13.874683 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0929 16:04:13.874868 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1409,"thread_start_us":1275,"threads_started":1}
I20260929 16:04:13.875411 22596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.864163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:35529 (local address 127.0.0.1:53780)
0929 16:04:13.866296 (+ 2133us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:13.866312 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:13.866428 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:13.867274 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:13.867279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:13.867308 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:13.867777 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.867792 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.869655 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.869661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.871922 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.871937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.873835 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.873844 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.874057 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.874087 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 16:04:13.874165 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 16:04:13.874291 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2020,"thread_start_us":707,"threads_started":1}
I20260929 16:04:13.878281 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.866633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:37907 (local address 127.0.0.1:57656)
0929 16:04:13.869380 (+ 2747us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:13.869410 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:13.869548 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:13.870246 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:13.870251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:13.870285 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:13.870777 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.870789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.872822 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.872829 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.875266 (+ 2437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:13.875281 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.877467 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.877473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:13.877657 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.877691 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 16:04:13.877781 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 16:04:13.877917 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2564,"thread_start_us":2416,"threads_started":1}
I20260929 16:04:13.878330 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:13.867198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.20.121.131:37907)
0929 16:04:13.868295 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:13.868303 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:13.869533 (+ 1230us) server_negotiation.cc:407] Connection header received
0929 16:04:13.869721 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:13.869729 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:13.870001 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:13.870224 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:13.871026 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.872672 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.875663 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:13.877249 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:13.877450 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:13.877638 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:13.877832 (+ 194us) server_negotiation.cc:299] Negotiation successful
0929 16:04:13.877985 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":928,"thread_start_us":759,"threads_started":1}
I20260929 16:04:13.879029 22374 tablet_service.cc:1467] Processing CreateTablet for tablet fc2a5a35a93d49a6bb439dc140b868d0 (DEFAULT_TABLE table=test-table [id=c506e98498ff41d28d1366b1d5e822aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:13.878989 22235 tablet_service.cc:1467] Processing CreateTablet for tablet fc2a5a35a93d49a6bb439dc140b868d0 (DEFAULT_TABLE table=test-table [id=c506e98498ff41d28d1366b1d5e822aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:13.880446 22235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2a5a35a93d49a6bb439dc140b868d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:13.880663 22374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2a5a35a93d49a6bb439dc140b868d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:13.882572 22513 tablet_service.cc:1467] Processing CreateTablet for tablet fc2a5a35a93d49a6bb439dc140b868d0 (DEFAULT_TABLE table=test-table [id=c506e98498ff41d28d1366b1d5e822aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:13.884053 22513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2a5a35a93d49a6bb439dc140b868d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:13.891722 22602 tablet_bootstrap.cc:492] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: Bootstrap starting.
I20260929 16:04:13.895502 22602 tablet_bootstrap.cc:654] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:13.896155 22603 tablet_bootstrap.cc:492] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: Bootstrap starting.
I20260929 16:04:13.897040 22602 log.cc:824] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:13.899698 22605 tablet_bootstrap.cc:492] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: Bootstrap starting.
I20260929 16:04:13.899864 22603 tablet_bootstrap.cc:654] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:13.900420 22602 tablet_bootstrap.cc:492] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: No bootstrap required, opened a new log
I20260929 16:04:13.900713 22602 ts_tablet_manager.cc:1397] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 16:04:13.901214 22603 log.cc:824] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:13.903294 22605 tablet_bootstrap.cc:654] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:13.904451 22605 log.cc:824] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:13.905153 22603 tablet_bootstrap.cc:492] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: No bootstrap required, opened a new log
I20260929 16:04:13.905433 22603 ts_tablet_manager.cc:1397] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260929 16:04:13.907097 22605 tablet_bootstrap.cc:492] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: No bootstrap required, opened a new log
I20260929 16:04:13.907356 22605 ts_tablet_manager.cc:1397] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 16:04:13.911852 22602 raft_consensus.cc:348] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:13.912358 22602 raft_consensus.cc:374] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:13.912472 22602 raft_consensus.cc:729] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca2699bb9334e37b8118616cce5b454, State: Initialized, Role: FOLLOWER
I20260929 16:04:13.913175 22602 consensus_queue.cc:260] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:13.915664 22605 raft_consensus.cc:348] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:13.916149 22605 raft_consensus.cc:374] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:13.916244 22605 raft_consensus.cc:729] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b5b235ed82f48a58a56341d93205261, State: Initialized, Role: FOLLOWER
I20260929 16:04:13.916999 22602 ts_tablet_manager.cc:1428] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: Time spent starting tablet: real 0.016s user 0.012s sys 0.005s
I20260929 16:04:13.916961 22605 consensus_queue.cc:260] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:13.917312 22603 raft_consensus.cc:348] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:13.918241 22603 raft_consensus.cc:374] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:13.918336 22603 raft_consensus.cc:729] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c937ec9283234120b42d86d3c7fe5161, State: Initialized, Role: FOLLOWER
I20260929 16:04:13.918700 22583 heartbeater.cc:503] Master 127.20.121.190:40505 was elected leader, sending a full tablet report...
I20260929 16:04:13.919384 22605 ts_tablet_manager.cc:1428] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260929 16:04:13.920137 22603 consensus_queue.cc:260] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:13.921388 22444 heartbeater.cc:503] Master 127.20.121.190:40505 was elected leader, sending a full tablet report...
I20260929 16:04:13.922256 22603 ts_tablet_manager.cc:1428] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
W20260929 16:04:13.926333 22306 tablet.cc:2369] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:04:13.985666 22445 tablet.cc:2369] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:04:14.029774 22584 tablet.cc:2369] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:04:14.063897 22610 raft_consensus.cc:482] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:14.064268 22610 raft_consensus.cc:504] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.066457 22610 leader_election.cc:288] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ca2699bb9334e37b8118616cce5b454 (127.20.121.129:40013), 9b5b235ed82f48a58a56341d93205261 (127.20.121.131:37907)
I20260929 16:04:14.073625 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.066879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:57327 (local address 127.20.121.129:40013)
0929 16:04:14.067121 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.067130 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.067657 (+ 527us) server_negotiation.cc:407] Connection header received
0929 16:04:14.067881 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.067888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.067974 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.068067 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:14.068744 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.069695 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.071656 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.072643 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.072761 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.072813 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.073236 (+ 423us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.073344 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 16:04:14.073655 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.066828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40013 (local address 127.20.121.130:57327)
0929 16:04:14.067522 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:14.067542 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:14.067735 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:14.068195 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:14.068198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:14.068222 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:14.068578 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.068589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.069819 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.069825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.071501 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.071512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.072839 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.072845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.073025 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.073063 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 16:04:14.073171 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 16:04:14.073299 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":408,"threads_started":1}
I20260929 16:04:14.074345 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.067919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:36215 (local address 127.20.121.131:37907)
0929 16:04:14.068113 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.068120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.068391 (+ 271us) server_negotiation.cc:407] Connection header received
0929 16:04:14.068561 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.068567 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.068642 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.068734 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:14.069407 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.070352 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.072283 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.073274 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.073415 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.073486 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.073890 (+ 404us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.074129 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 16:04:14.074689 22255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2a5a35a93d49a6bb439dc140b868d0" candidate_uuid: "c937ec9283234120b42d86d3c7fe5161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca2699bb9334e37b8118616cce5b454" is_pre_election: true
I20260929 16:04:14.074898 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.067689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:37907 (local address 127.20.121.130:36215)
0929 16:04:14.068298 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:14.068314 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:14.068423 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:14.068754 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:14.068758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:14.068781 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:14.069235 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.069246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.070502 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.070506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.072140 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.072153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.073496 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.073502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.073693 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.073726 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 16:04:14.074522 (+ 796us) client_negotiation.cc:240] Negotiation successful
0929 16:04:14.074648 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"spinlock_wait_cycles":1408,"thread_start_us":296,"threads_started":1}
I20260929 16:04:14.075220 22255 raft_consensus.cc:2454] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c937ec9283234120b42d86d3c7fe5161 in term 0.
I20260929 16:04:14.075820 22327 leader_election.cc:302] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca2699bb9334e37b8118616cce5b454, c937ec9283234120b42d86d3c7fe5161; no voters:
I20260929 16:04:14.075847 22533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2a5a35a93d49a6bb439dc140b868d0" candidate_uuid: "c937ec9283234120b42d86d3c7fe5161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b5b235ed82f48a58a56341d93205261" is_pre_election: true
I20260929 16:04:14.076373 22533 raft_consensus.cc:2454] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c937ec9283234120b42d86d3c7fe5161 in term 0.
I20260929 16:04:14.076630 22610 raft_consensus.cc:2781] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:14.076776 22610 raft_consensus.cc:482] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:14.076893 22610 raft_consensus.cc:3037] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:14.079053 22610 raft_consensus.cc:504] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.079668 22610 leader_election.cc:288] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [CANDIDATE]: Term 1 election: Requested vote from peers 0ca2699bb9334e37b8118616cce5b454 (127.20.121.129:40013), 9b5b235ed82f48a58a56341d93205261 (127.20.121.131:37907)
I20260929 16:04:14.080269 22255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2a5a35a93d49a6bb439dc140b868d0" candidate_uuid: "c937ec9283234120b42d86d3c7fe5161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca2699bb9334e37b8118616cce5b454"
I20260929 16:04:14.080302 22533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2a5a35a93d49a6bb439dc140b868d0" candidate_uuid: "c937ec9283234120b42d86d3c7fe5161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b5b235ed82f48a58a56341d93205261"
I20260929 16:04:14.080459 22255 raft_consensus.cc:3037] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:14.080533 22533 raft_consensus.cc:3037] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:14.083021 22533 raft_consensus.cc:2454] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c937ec9283234120b42d86d3c7fe5161 in term 1.
I20260929 16:04:14.083029 22255 raft_consensus.cc:2454] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c937ec9283234120b42d86d3c7fe5161 in term 1.
I20260929 16:04:14.083410 22327 leader_election.cc:302] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b5b235ed82f48a58a56341d93205261, c937ec9283234120b42d86d3c7fe5161; no voters:
I20260929 16:04:14.083690 22610 raft_consensus.cc:2781] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:14.084128 22610 raft_consensus.cc:686] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 1 LEADER]: Becoming Leader. State: Replica: c937ec9283234120b42d86d3c7fe5161, State: Running, Role: LEADER
I20260929 16:04:14.084571 22610 consensus_queue.cc:237] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.089753 22098 catalog_manager.cc:5495] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 reported cstate change: term changed from 0 to 1, leader changed from <none> to c937ec9283234120b42d86d3c7fe5161 (127.20.121.130). New cstate: current_term: 1 leader_uuid: "c937ec9283234120b42d86d3c7fe5161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } health_report { overall_health: UNKNOWN } } }
I20260929 16:04:14.123703 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.117078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.20.121.130:35529)
0929 16:04:14.117356 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.117364 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.117473 (+ 109us) server_negotiation.cc:407] Connection header received
0929 16:04:14.117853 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.117857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.117946 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.118144 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:14.118529 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.119302 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.121119 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.121613 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.121764 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.122425 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:14.123026 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:14.123193 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.123313 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.123411 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260929 16:04:14.132942 22533 raft_consensus.cc:1261] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261 [term 1 FOLLOWER]: Refusing update from remote peer c937ec9283234120b42d86d3c7fe5161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:14.132941 22255 raft_consensus.cc:1261] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454 [term 1 FOLLOWER]: Refusing update from remote peer c937ec9283234120b42d86d3c7fe5161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:14.133948 22613 consensus_queue.cc:1035] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:14.134439 22610 consensus_queue.cc:1035] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:14.160009 22618 mvcc.cc:205] Tried to move back new op lower bound from 7334698410517258240 to 7334698410334093312. Current Snapshot: MvccSnapshot[applied={T|T < 7334698410517258240}]
I20260929 16:04:14.160804 22617 mvcc.cc:205] Tried to move back new op lower bound from 7334698410517258240 to 7334698410334093312. Current Snapshot: MvccSnapshot[applied={T|T < 7334698410517258240}]
I20260929 16:04:14.173877 22098 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:14.174259 22098 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:14.178391 22098 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737: Sending DeleteTablet for 3 replicas of tablet fc2a5a35a93d49a6bb439dc140b868d0
I20260929 16:04:14.180111 22235 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2a5a35a93d49a6bb439dc140b868d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:14 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:55744
I20260929 16:04:14.180363 22374 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2a5a35a93d49a6bb439dc140b868d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:14 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:53780
I20260929 16:04:14.181077 22513 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2a5a35a93d49a6bb439dc140b868d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:14 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:57656
I20260929 16:04:14.181842 22098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:04:14.183251 22098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:14.194526 22628 tablet_replica.cc:330] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: stopping tablet replica
I20260929 16:04:14.195147 22628 raft_consensus.cc:2229] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:14.195768 22628 raft_consensus.cc:2258] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:14.197949 22373 tablet_service.cc:1467] Processing CreateTablet for tablet d4860fbced9842569f55603181307a41 (DEFAULT_TABLE table=test-table [id=09c41ec15fa84aa183ea9ce82bb71f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:14.198760 22373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4860fbced9842569f55603181307a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:14.198825 22234 tablet_service.cc:1467] Processing CreateTablet for tablet d4860fbced9842569f55603181307a41 (DEFAULT_TABLE table=test-table [id=09c41ec15fa84aa183ea9ce82bb71f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:14.199352 22234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4860fbced9842569f55603181307a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:14.201941 22626 tablet_replica.cc:330] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: stopping tablet replica
I20260929 16:04:14.202538 22626 raft_consensus.cc:2229] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:14.203167 22626 raft_consensus.cc:2258] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:14.205365 22627 tablet_replica.cc:330] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: stopping tablet replica
I20260929 16:04:14.205955 22627 raft_consensus.cc:2229] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:04:14.206599 22603 tablet_bootstrap.cc:492] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161: Bootstrap starting.
I20260929 16:04:14.206629 22628 ts_tablet_manager.cc:1905] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:14.208568 22513 tablet_service.cc:1467] Processing CreateTablet for tablet d4860fbced9842569f55603181307a41 (DEFAULT_TABLE table=test-table [id=09c41ec15fa84aa183ea9ce82bb71f0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:14.208799 22603 tablet_bootstrap.cc:654] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:14.209110 22513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4860fbced9842569f55603181307a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:14.210278 22602 tablet_bootstrap.cc:492] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454: Bootstrap starting.
I20260929 16:04:14.207444 22627 raft_consensus.cc:2258] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:14.212051 22602 tablet_bootstrap.cc:654] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:14.215740 22628 ts_tablet_manager.cc:1918] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:04:14.215955 22628 log.cc:1196] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/fc2a5a35a93d49a6bb439dc140b868d0
I20260929 16:04:14.216562 22628 ts_tablet_manager.cc:1939] T fc2a5a35a93d49a6bb439dc140b868d0 P 9b5b235ed82f48a58a56341d93205261: Deleting consensus metadata
I20260929 16:04:14.217334 22605 tablet_bootstrap.cc:492] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261: Bootstrap starting.
I20260929 16:04:14.219062 22605 tablet_bootstrap.cc:654] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:14.219236 22626 ts_tablet_manager.cc:1905] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:14.221853 22627 ts_tablet_manager.cc:1905] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:14.227463 22603 tablet_bootstrap.cc:492] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161: No bootstrap required, opened a new log
I20260929 16:04:14.227670 22603 ts_tablet_manager.cc:1397] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.007s
I20260929 16:04:14.228641 22603 raft_consensus.cc:348] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.228914 22603 raft_consensus.cc:374] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:14.229033 22603 raft_consensus.cc:729] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c937ec9283234120b42d86d3c7fe5161, State: Initialized, Role: FOLLOWER
I20260929 16:04:14.229359 22603 consensus_queue.cc:260] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.229979 22603 ts_tablet_manager.cc:1428] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260929 16:04:14.231453 22084 catalog_manager.cc:4841] TS 9b5b235ed82f48a58a56341d93205261 (127.20.121.131:37907): tablet fc2a5a35a93d49a6bb439dc140b868d0 (table test-table [id=c506e98498ff41d28d1366b1d5e822aa]) successfully deleted
I20260929 16:04:14.236096 22605 tablet_bootstrap.cc:492] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261: No bootstrap required, opened a new log
I20260929 16:04:14.236299 22605 ts_tablet_manager.cc:1397] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.001s
I20260929 16:04:14.237347 22605 raft_consensus.cc:348] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.237605 22605 raft_consensus.cc:374] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:14.237707 22605 raft_consensus.cc:729] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b5b235ed82f48a58a56341d93205261, State: Initialized, Role: FOLLOWER
I20260929 16:04:14.238056 22605 consensus_queue.cc:260] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.238600 22602 tablet_bootstrap.cc:492] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454: No bootstrap required, opened a new log
I20260929 16:04:14.239785 22605 ts_tablet_manager.cc:1428] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 16:04:14.238795 22602 ts_tablet_manager.cc:1397] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454: Time spent bootstrapping tablet: real 0.029s user 0.007s sys 0.003s
I20260929 16:04:14.241178 22602 raft_consensus.cc:348] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.241505 22602 raft_consensus.cc:374] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:14.241636 22602 raft_consensus.cc:729] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca2699bb9334e37b8118616cce5b454, State: Initialized, Role: FOLLOWER
I20260929 16:04:14.242069 22602 consensus_queue.cc:260] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.242640 22626 ts_tablet_manager.cc:1918] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:04:14.242833 22626 log.cc:1196] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/fc2a5a35a93d49a6bb439dc140b868d0
I20260929 16:04:14.242964 22602 ts_tablet_manager.cc:1428] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 16:04:14.243650 22626 ts_tablet_manager.cc:1939] T fc2a5a35a93d49a6bb439dc140b868d0 P 0ca2699bb9334e37b8118616cce5b454: Deleting consensus metadata
I20260929 16:04:14.246160 22084 catalog_manager.cc:4841] TS 0ca2699bb9334e37b8118616cce5b454 (127.20.121.129:40013): tablet fc2a5a35a93d49a6bb439dc140b868d0 (table test-table [id=c506e98498ff41d28d1366b1d5e822aa]) successfully deleted
I20260929 16:04:14.247481 22627 ts_tablet_manager.cc:1918] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:04:14.248374 22627 log.cc:1196] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/fc2a5a35a93d49a6bb439dc140b868d0
I20260929 16:04:14.249450 22627 ts_tablet_manager.cc:1939] T fc2a5a35a93d49a6bb439dc140b868d0 P c937ec9283234120b42d86d3c7fe5161: Deleting consensus metadata
I20260929 16:04:14.251367 22085 catalog_manager.cc:4841] TS c937ec9283234120b42d86d3c7fe5161 (127.20.121.130:35529): tablet fc2a5a35a93d49a6bb439dc140b868d0 (table test-table [id=c506e98498ff41d28d1366b1d5e822aa]) successfully deleted
I20260929 16:04:14.252331 22609 raft_consensus.cc:482] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:14.252530 22609 raft_consensus.cc:504] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.254335 22609 leader_election.cc:288] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ca2699bb9334e37b8118616cce5b454 (127.20.121.129:40013), c937ec9283234120b42d86d3c7fe5161 (127.20.121.130:35529)
I20260929 16:04:14.274375 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.255048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:49177 (local address 127.20.121.130:35529)
0929 16:04:14.255213 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.255221 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.266775 (+ 11554us) server_negotiation.cc:407] Connection header received
0929 16:04:14.267032 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.267039 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.267113 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.267175 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:14.268110 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.269365 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.271818 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.273216 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.273418 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.273502 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.273875 (+ 373us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.274059 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 16:04:14.274644 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.254913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:45253 (local address 127.20.121.129:40013)
0929 16:04:14.255084 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.255091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.265983 (+ 10892us) server_negotiation.cc:407] Connection header received
0929 16:04:14.266860 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.266867 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.266948 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.267013 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:14.267861 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.269013 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.272037 (+ 3024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.273281 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.273447 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.273524 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.274198 (+ 674us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.274341 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 16:04:14.274428 22633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.254736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:35529 (local address 127.20.121.131:49177)
0929 16:04:14.266668 (+ 11932us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:14.266691 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:14.266878 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:14.267399 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:14.267404 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:14.267429 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:14.267897 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.267908 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.269570 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.269575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.271558 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.271582 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.273419 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.273426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.273672 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.273721 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 16:04:14.273835 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 16:04:14.273990 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11805,"mutex_wait_us":29,"thread_start_us":11607,"threads_started":1}
I20260929 16:04:14.275234 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.254530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40013 (local address 127.20.121.131:45253)
0929 16:04:14.265837 (+ 11307us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:14.265864 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:14.266647 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:14.267220 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:14.267225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:14.267254 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:14.267649 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.267662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.269196 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.269201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.271567 (+ 2366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.271871 (+ 304us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.273734 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.273740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.273981 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.274037 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 16:04:14.274832 (+ 795us) client_negotiation.cc:240] Negotiation successful
0929 16:04:14.274989 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10657,"mutex_wait_us":1,"thread_start_us":11495,"threads_started":1}
I20260929 16:04:14.275992 22255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4860fbced9842569f55603181307a41" candidate_uuid: "9b5b235ed82f48a58a56341d93205261" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca2699bb9334e37b8118616cce5b454" is_pre_election: true
I20260929 16:04:14.276259 22255 raft_consensus.cc:2454] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b5b235ed82f48a58a56341d93205261 in term 0.
I20260929 16:04:14.276144 22394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4860fbced9842569f55603181307a41" candidate_uuid: "9b5b235ed82f48a58a56341d93205261" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c937ec9283234120b42d86d3c7fe5161" is_pre_election: true
I20260929 16:04:14.276753 22466 leader_election.cc:302] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca2699bb9334e37b8118616cce5b454, 9b5b235ed82f48a58a56341d93205261; no voters:
I20260929 16:04:14.276605 22394 raft_consensus.cc:2454] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b5b235ed82f48a58a56341d93205261 in term 0.
I20260929 16:04:14.277179 22609 raft_consensus.cc:2781] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:14.277304 22609 raft_consensus.cc:482] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:14.277436 22609 raft_consensus.cc:3037] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:14.279767 22609 raft_consensus.cc:504] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.280442 22609 leader_election.cc:288] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [CANDIDATE]: Term 1 election: Requested vote from peers 0ca2699bb9334e37b8118616cce5b454 (127.20.121.129:40013), c937ec9283234120b42d86d3c7fe5161 (127.20.121.130:35529)
I20260929 16:04:14.280920 22255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4860fbced9842569f55603181307a41" candidate_uuid: "9b5b235ed82f48a58a56341d93205261" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca2699bb9334e37b8118616cce5b454"
I20260929 16:04:14.281011 22394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4860fbced9842569f55603181307a41" candidate_uuid: "9b5b235ed82f48a58a56341d93205261" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c937ec9283234120b42d86d3c7fe5161"
I20260929 16:04:14.281147 22255 raft_consensus.cc:3037] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:14.281220 22394 raft_consensus.cc:3037] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:14.283726 22255 raft_consensus.cc:2454] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b5b235ed82f48a58a56341d93205261 in term 1.
I20260929 16:04:14.284192 22466 leader_election.cc:302] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca2699bb9334e37b8118616cce5b454, 9b5b235ed82f48a58a56341d93205261; no voters:
I20260929 16:04:14.284488 22609 raft_consensus.cc:2781] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:14.284566 22394 raft_consensus.cc:2454] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b5b235ed82f48a58a56341d93205261 in term 1.
I20260929 16:04:14.284951 22609 raft_consensus.cc:686] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 1 LEADER]: Becoming Leader. State: Replica: 9b5b235ed82f48a58a56341d93205261, State: Running, Role: LEADER
I20260929 16:04:14.285380 22609 consensus_queue.cc:237] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.289565 22097 catalog_manager.cc:5495] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b5b235ed82f48a58a56341d93205261 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "9b5b235ed82f48a58a56341d93205261" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } health_report { overall_health: HEALTHY } } }
Sep 29 16:04:14 dist-test-slave-x07d krb5kdc[22053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697851, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:14.353950 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.336928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56970 (local address 127.20.121.190:40505)
0929 16:04:14.337147 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.337156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.337690 (+ 534us) server_negotiation.cc:407] Connection header received
0929 16:04:14.337930 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.337936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.338090 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.338204 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:14.339801 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.341023 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.343109 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.343732 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.343851 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.348230 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:14.348236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:14.348239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:14.348288 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:14.351156 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.351824 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.351831 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.351834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.351967 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.352329 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.352334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.352337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.352624 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:14.352773 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.353168 (+ 395us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.353495 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":65}
I20260929 16:04:14.353950 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.336849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40505 (local address 127.0.0.1:56970)
0929 16:04:14.337541 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:14.337572 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:14.337744 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:14.338247 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:14.338252 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:14.339219 (+ 967us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:14.339581 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.339598 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.341200 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.341206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.342902 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.342918 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.343929 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.343935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.344105 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.344960 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:14.344985 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:14.348000 (+ 3015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:14.351288 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:14.351303 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:14.351311 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:14.351705 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:14.352072 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:14.352078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:14.352081 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:14.352221 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:14.352750 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:14.352758 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:14.352935 (+ 177us) client_negotiation.cc:769] Sending connection context
0929 16:04:14.353088 (+ 153us) client_negotiation.cc:240] Negotiation successful
0929 16:04:14.353381 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":362,"threads_started":1}
I20260929 16:04:14.360534 22097 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:56970:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 16:04:14.361649 22097 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:14.369107 22234 tablet_service.cc:1467] Processing CreateTablet for tablet 3a563b34e4e44356adede9e1c571dd18 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e3e03bfd17d4f789d0b7493aef6011a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 16:04:14.369148 22513 tablet_service.cc:1467] Processing CreateTablet for tablet 3a563b34e4e44356adede9e1c571dd18 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e3e03bfd17d4f789d0b7493aef6011a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 16:04:14.369226 22374 tablet_service.cc:1467] Processing CreateTablet for tablet 3a563b34e4e44356adede9e1c571dd18 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6e3e03bfd17d4f789d0b7493aef6011a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 16:04:14.369699 22234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a563b34e4e44356adede9e1c571dd18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:14.369720 22513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a563b34e4e44356adede9e1c571dd18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:14.369794 22374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a563b34e4e44356adede9e1c571dd18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:14.377974 22605 tablet_bootstrap.cc:492] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: Bootstrap starting.
I20260929 16:04:14.377966 22602 tablet_bootstrap.cc:492] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: Bootstrap starting.
I20260929 16:04:14.378439 22603 tablet_bootstrap.cc:492] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: Bootstrap starting.
I20260929 16:04:14.379987 22602 tablet_bootstrap.cc:654] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:14.379998 22605 tablet_bootstrap.cc:654] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:14.380218 22603 tablet_bootstrap.cc:654] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:14.391588 22605 tablet_bootstrap.cc:492] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: No bootstrap required, opened a new log
I20260929 16:04:14.391806 22605 ts_tablet_manager.cc:1397] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260929 16:04:14.392299 22602 tablet_bootstrap.cc:492] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: No bootstrap required, opened a new log
I20260929 16:04:14.392493 22602 ts_tablet_manager.cc:1397] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260929 16:04:14.392705 22605 raft_consensus.cc:348] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.392987 22605 raft_consensus.cc:374] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:14.393086 22605 raft_consensus.cc:729] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b5b235ed82f48a58a56341d93205261, State: Initialized, Role: FOLLOWER
I20260929 16:04:14.393289 22602 raft_consensus.cc:348] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.393390 22605 consensus_queue.cc:260] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.393563 22602 raft_consensus.cc:374] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:14.393639 22602 raft_consensus.cc:729] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca2699bb9334e37b8118616cce5b454, State: Initialized, Role: FOLLOWER
I20260929 16:04:14.393875 22609 tablet_replica.cc:439] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } } }
I20260929 16:04:14.393956 22602 consensus_queue.cc:260] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.394243 22609 tablet_replica.cc:442] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:14.394487 22608 tablet_replica.cc:439] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } } }
I20260929 16:04:14.394841 22608 tablet_replica.cc:442] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:14.395406 22605 ts_tablet_manager.cc:1428] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 16:04:14.396579 22602 ts_tablet_manager.cc:1428] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260929 16:04:14.398325 22603 tablet_bootstrap.cc:492] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: No bootstrap required, opened a new log
I20260929 16:04:14.399161 22603 ts_tablet_manager.cc:1397] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.008s
I20260929 16:04:14.400059 22603 raft_consensus.cc:348] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.400406 22603 raft_consensus.cc:374] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:14.400569 22603 raft_consensus.cc:729] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c937ec9283234120b42d86d3c7fe5161, State: Initialized, Role: FOLLOWER
I20260929 16:04:14.400803 22603 consensus_queue.cc:260] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.401306 22622 tablet_replica.cc:439] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } } }
I20260929 16:04:14.401654 22622 tablet_replica.cc:442] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:14.401937 22603 ts_tablet_manager.cc:1428] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260929 16:04:14.506918 22608 raft_consensus.cc:482] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:14.507211 22608 raft_consensus.cc:504] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.509539 22608 leader_election.cc:288] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c937ec9283234120b42d86d3c7fe5161 (127.20.121.130:35529), 9b5b235ed82f48a58a56341d93205261 (127.20.121.131:37907)
I20260929 16:04:14.518430 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.510000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:56053 (local address 127.20.121.130:35529)
0929 16:04:14.510239 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.510245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.511332 (+ 1087us) server_negotiation.cc:407] Connection header received
0929 16:04:14.511674 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.511687 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.511779 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.511847 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:14.512652 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.513791 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.516363 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.517332 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.517507 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.517580 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.517941 (+ 361us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.518129 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 16:04:14.518482 22646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.509850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:35529 (local address 127.20.121.129:56053)
0929 16:04:14.511206 (+ 1356us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:14.511228 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:14.511486 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:14.512061 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:14.512067 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:14.512092 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:14.512476 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.512489 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.513926 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.513931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.516185 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.516214 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.517565 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.517572 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.517744 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.517779 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 16:04:14.517902 (+ 123us) client_negotiation.cc:240] Negotiation successful
0929 16:04:14.518067 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1216,"thread_start_us":1479,"threads_started":1}
I20260929 16:04:14.519493 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.509900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:37907 (local address 127.20.121.129:59881)
0929 16:04:14.511643 (+ 1743us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:14.511660 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:14.511848 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:14.512742 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:14.512750 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:14.512786 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:14.513051 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.513058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.514335 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.514341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.516996 (+ 2655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.517011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.518746 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.518754 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.518903 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.518925 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 16:04:14.519026 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 16:04:14.519169 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1642,"thread_start_us":977,"threads_started":1}
I20260929 16:04:14.519542 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.510455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:59881 (local address 127.20.121.131:37907)
0929 16:04:14.510637 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.510644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.511744 (+ 1100us) server_negotiation.cc:407] Connection header received
0929 16:04:14.512000 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.512005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.512101 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.512168 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:14.513219 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.514200 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.517177 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.518324 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.518496 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.518572 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.519109 (+ 537us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.519261 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:04:14.519480 22394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a563b34e4e44356adede9e1c571dd18" candidate_uuid: "0ca2699bb9334e37b8118616cce5b454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c937ec9283234120b42d86d3c7fe5161" is_pre_election: true
I20260929 16:04:14.519783 22394 raft_consensus.cc:2454] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ca2699bb9334e37b8118616cce5b454 in term 0.
I20260929 16:04:14.520061 22533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a563b34e4e44356adede9e1c571dd18" candidate_uuid: "0ca2699bb9334e37b8118616cce5b454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b5b235ed82f48a58a56341d93205261" is_pre_election: true
I20260929 16:04:14.520314 22533 raft_consensus.cc:2454] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ca2699bb9334e37b8118616cce5b454 in term 0.
I20260929 16:04:14.520453 22188 leader_election.cc:302] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca2699bb9334e37b8118616cce5b454, c937ec9283234120b42d86d3c7fe5161; no voters:
I20260929 16:04:14.520886 22608 raft_consensus.cc:2781] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:14.521008 22608 raft_consensus.cc:482] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:14.521095 22608 raft_consensus.cc:3037] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:14.525092 22608 raft_consensus.cc:504] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.525833 22608 leader_election.cc:288] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [CANDIDATE]: Term 1 election: Requested vote from peers c937ec9283234120b42d86d3c7fe5161 (127.20.121.130:35529), 9b5b235ed82f48a58a56341d93205261 (127.20.121.131:37907)
I20260929 16:04:14.526374 22533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a563b34e4e44356adede9e1c571dd18" candidate_uuid: "0ca2699bb9334e37b8118616cce5b454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b5b235ed82f48a58a56341d93205261"
I20260929 16:04:14.526367 22394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a563b34e4e44356adede9e1c571dd18" candidate_uuid: "0ca2699bb9334e37b8118616cce5b454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c937ec9283234120b42d86d3c7fe5161"
I20260929 16:04:14.526583 22533 raft_consensus.cc:3037] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:14.526610 22394 raft_consensus.cc:3037] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:14.528699 22533 raft_consensus.cc:2454] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ca2699bb9334e37b8118616cce5b454 in term 1.
I20260929 16:04:14.529157 22187 leader_election.cc:302] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca2699bb9334e37b8118616cce5b454, 9b5b235ed82f48a58a56341d93205261; no voters:
I20260929 16:04:14.529461 22608 raft_consensus.cc:2781] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:14.529803 22394 raft_consensus.cc:2454] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ca2699bb9334e37b8118616cce5b454 in term 1.
I20260929 16:04:14.530071 22608 raft_consensus.cc:686] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [term 1 LEADER]: Becoming Leader. State: Replica: 0ca2699bb9334e37b8118616cce5b454, State: Running, Role: LEADER
I20260929 16:04:14.530695 22608 consensus_queue.cc:237] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } }
I20260929 16:04:14.532773 22647 tablet_replica.cc:439] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: TxnStatusTablet state changed. Reason: New leader 0ca2699bb9334e37b8118616cce5b454. Latest consensus state: current_term: 1 leader_uuid: "0ca2699bb9334e37b8118616cce5b454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } } }
I20260929 16:04:14.533689 22647 tablet_replica.cc:442] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: This TxnStatusTablet replica's current role is: LEADER
I20260929 16:04:14.534349 22649 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 16:04:14.534597 22649 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 16:04:14.534677 22097 catalog_manager.cc:5495] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ca2699bb9334e37b8118616cce5b454 (127.20.121.129). New cstate: current_term: 1 leader_uuid: "0ca2699bb9334e37b8118616cce5b454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } health_report { overall_health: UNKNOWN } } }
I20260929 16:04:14.560240 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.550859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40013 (local address 127.0.0.1:55754)
0929 16:04:14.551218 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:14.551241 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:14.551392 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:14.552279 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:14.552285 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:14.552297 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:04:14.552483 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.552495 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.554004 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.554044 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.556718 (+ 2674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.556738 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.557907 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.557913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.558164 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.558250 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:04:14.559583 (+ 1333us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:04:14.559600 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 16:04:14.559702 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 16:04:14.559888 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260929 16:04:14.560957 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.550928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55754 (local address 127.20.121.129:40013)
0929 16:04:14.551672 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.551679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.551697 (+ 18us) server_negotiation.cc:407] Connection header received
0929 16:04:14.551798 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.551804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.551921 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.552056 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:14.552647 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.553821 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.556924 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.557629 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.557763 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.558499 (+ 736us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:14.559286 (+ 787us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:14.559425 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.560561 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.560656 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619}
I20260929 16:04:14.565748 22533 raft_consensus.cc:1261] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261 [term 1 FOLLOWER]: Refusing update from remote peer 0ca2699bb9334e37b8118616cce5b454: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:14.566514 22394 raft_consensus.cc:1261] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161 [term 1 FOLLOWER]: Refusing update from remote peer 0ca2699bb9334e37b8118616cce5b454: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:14.566535 22608 consensus_queue.cc:1035] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:14.567284 22608 consensus_queue.cc:1035] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:14.571617 22609 tablet_replica.cc:439] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: TxnStatusTablet state changed. Reason: New leader 0ca2699bb9334e37b8118616cce5b454. Latest consensus state: current_term: 1 leader_uuid: "0ca2699bb9334e37b8118616cce5b454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } } }
I20260929 16:04:14.571923 22609 tablet_replica.cc:442] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:14.573370 22608 tablet_replica.cc:439] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ca2699bb9334e37b8118616cce5b454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } } }
I20260929 16:04:14.573719 22608 tablet_replica.cc:442] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: This TxnStatusTablet replica's current role is: LEADER
I20260929 16:04:14.574311 22609 tablet_replica.cc:439] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ca2699bb9334e37b8118616cce5b454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } } }
I20260929 16:04:14.574991 22609 tablet_replica.cc:442] T 3a563b34e4e44356adede9e1c571dd18 P 9b5b235ed82f48a58a56341d93205261: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:14.589059 22647 tablet_replica.cc:439] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ca2699bb9334e37b8118616cce5b454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } } }
I20260929 16:04:14.589377 22647 tablet_replica.cc:442] T 3a563b34e4e44356adede9e1c571dd18 P 0ca2699bb9334e37b8118616cce5b454: This TxnStatusTablet replica's current role is: LEADER
I20260929 16:04:14.591614 22622 tablet_replica.cc:439] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: TxnStatusTablet state changed. Reason: New leader 0ca2699bb9334e37b8118616cce5b454. Latest consensus state: current_term: 1 leader_uuid: "0ca2699bb9334e37b8118616cce5b454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } } }
I20260929 16:04:14.591989 22622 tablet_replica.cc:442] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:14.592418 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.583908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.20.121.131:37907)
0929 16:04:14.584109 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.584116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.584137 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:04:14.584248 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.584253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.584385 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.584625 (+ 240us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:14.585291 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.586381 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.589022 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.589689 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.589827 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.590556 (+ 729us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:14.591274 (+ 718us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:14.591427 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.591879 (+ 452us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.592120 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 16:04:14.593554 22622 tablet_replica.cc:439] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ca2699bb9334e37b8118616cce5b454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 } } peers { permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 } } peers { permanent_uuid: "9b5b235ed82f48a58a56341d93205261" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37907 } } }
I20260929 16:04:14.593772 22622 tablet_replica.cc:442] T 3a563b34e4e44356adede9e1c571dd18 P c937ec9283234120b42d86d3c7fe5161: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:14.614147 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.606145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:49937 (local address 127.20.121.129:40013)
0929 16:04:14.606349 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.606355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.607051 (+ 696us) server_negotiation.cc:407] Connection header received
0929 16:04:14.607284 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.607290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.607378 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.607482 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:14.608132 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.609271 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.611377 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.612192 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.612329 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.613214 (+ 885us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:14.613366 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:14.613466 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.613641 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.613729 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 16:04:14.614265 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.606150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40013 (local address 127.20.121.131:49937)
0929 16:04:14.606904 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:14.606928 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:14.607113 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:14.607528 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:14.607533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:14.607548 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:04:14.607935 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.607950 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.609431 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.609439 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.611199 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.611220 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.612729 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.612736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.612917 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.613000 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:04:14.613488 (+ 488us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:04:14.613500 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 16:04:14.613608 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 16:04:14.613833 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":392,"threads_started":1}
I20260929 16:04:14.629770 22394 raft_consensus.cc:1261] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161 [term 1 FOLLOWER]: Refusing update from remote peer 9b5b235ed82f48a58a56341d93205261: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:14.629873 22255 raft_consensus.cc:1261] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454 [term 1 FOLLOWER]: Refusing update from remote peer 9b5b235ed82f48a58a56341d93205261: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:14.630759 22609 consensus_queue.cc:1035] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca2699bb9334e37b8118616cce5b454" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:14.631114 22634 consensus_queue.cc:1035] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c937ec9283234120b42d86d3c7fe5161" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:14.676789 22616 mvcc.cc:205] Tried to move back new op lower bound from 7334698412554891264 to 7334698411156578304. Current Snapshot: MvccSnapshot[applied={T|T < 7334698412554891264}]
I20260929 16:04:14.703464 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.692348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:47031 (local address 127.20.121.131:37907)
0929 16:04:14.692548 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.692555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.696388 (+ 3833us) server_negotiation.cc:407] Connection header received
0929 16:04:14.696601 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.696607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.696691 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.696787 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:14.697543 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.698635 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.701016 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.701769 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.701870 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.702612 (+ 742us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:14.702756 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:14.702818 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.703035 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.703140 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 16:04:14.704353 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.692393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:37907 (local address 127.20.121.129:47031)
0929 16:04:14.696261 (+ 3868us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:14.696286 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:14.696455 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:14.696899 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:14.696905 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:14.696921 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:04:14.697364 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.697377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.698810 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.698816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.700832 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:14.700849 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.702115 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.702121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:14.702319 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.702397 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:04:14.702895 (+ 498us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:04:14.702905 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 16:04:14.703741 (+ 836us) client_negotiation.cc:240] Negotiation successful
0929 16:04:14.703897 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3231,"thread_start_us":3043,"threads_started":1}
I20260929 16:04:14.791685 22097 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:14.791985 22097 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:14.796065 22097 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9886447215454108a7b672135c64d737: Sending DeleteTablet for 3 replicas of tablet d4860fbced9842569f55603181307a41
I20260929 16:04:14.797788 22234 tablet_service.cc:1514] Processing DeleteTablet for tablet d4860fbced9842569f55603181307a41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:14 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:55744
I20260929 16:04:14.798360 22374 tablet_service.cc:1514] Processing DeleteTablet for tablet d4860fbced9842569f55603181307a41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:14 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:53780
I20260929 16:04:14.799089 22513 tablet_service.cc:1514] Processing DeleteTablet for tablet d4860fbced9842569f55603181307a41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:14 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:57656
I20260929 16:04:14.804979 22667 tablet_replica.cc:330] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161: stopping tablet replica
I20260929 16:04:14.805248 22667 raft_consensus.cc:2229] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:14.805526 22667 raft_consensus.cc:2258] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:14.807019 22667 ts_tablet_manager.cc:1905] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:14.807541 22668 tablet_replica.cc:330] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261: stopping tablet replica
I20260929 16:04:14.807852 22668 raft_consensus.cc:2229] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:04:14.808640 22668 raft_consensus.cc:2258] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:14.810986 22668 ts_tablet_manager.cc:1905] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:14.814796 22667 ts_tablet_manager.cc:1918] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:04:14.814971 22667 log.cc:1196] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/d4860fbced9842569f55603181307a41
I20260929 16:04:14.815490 22667 ts_tablet_manager.cc:1939] T d4860fbced9842569f55603181307a41 P c937ec9283234120b42d86d3c7fe5161: Deleting consensus metadata
I20260929 16:04:14.816813 22085 catalog_manager.cc:4841] TS c937ec9283234120b42d86d3c7fe5161 (127.20.121.130:35529): tablet d4860fbced9842569f55603181307a41 (table test-table [id=09c41ec15fa84aa183ea9ce82bb71f0d]) successfully deleted
I20260929 16:04:14.828881 22666 tablet_replica.cc:330] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454: stopping tablet replica
I20260929 16:04:14.829169 22666 raft_consensus.cc:2229] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:14.829495 22666 raft_consensus.cc:2258] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:14.838157 22666 ts_tablet_manager.cc:1905] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:14.838543 22668 ts_tablet_manager.cc:1918] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:04:14.838804 22668 log.cc:1196] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/d4860fbced9842569f55603181307a41
I20260929 16:04:14.839428 22668 ts_tablet_manager.cc:1939] T d4860fbced9842569f55603181307a41 P 9b5b235ed82f48a58a56341d93205261: Deleting consensus metadata
I20260929 16:04:14.841086 22084 catalog_manager.cc:4841] TS 9b5b235ed82f48a58a56341d93205261 (127.20.121.131:37907): tablet d4860fbced9842569f55603181307a41 (table test-table [id=09c41ec15fa84aa183ea9ce82bb71f0d]) successfully deleted
Sep 29 16:04:14 dist-test-slave-x07d krb5kdc[22053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697853, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:14.849623 22666 ts_tablet_manager.cc:1918] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:04:14.849805 22666 log.cc:1196] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/d4860fbced9842569f55603181307a41
I20260929 16:04:14.850456 22666 ts_tablet_manager.cc:1939] T d4860fbced9842569f55603181307a41 P 0ca2699bb9334e37b8118616cce5b454: Deleting consensus metadata
I20260929 16:04:14.852051 22084 catalog_manager.cc:4841] TS 0ca2699bb9334e37b8118616cce5b454 (127.20.121.129:40013): tablet d4860fbced9842569f55603181307a41 (table test-table [id=09c41ec15fa84aa183ea9ce82bb71f0d]) successfully deleted
I20260929 16:04:14.860754 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.809642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55770 (local address 127.20.121.129:40013)
0929 16:04:14.810154 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.810161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.830418 (+ 20257us) server_negotiation.cc:407] Connection header received
0929 16:04:14.834193 (+ 3775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.834202 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.834325 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.834413 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:14.838509 (+ 4096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.839828 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.842903 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.843732 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.843857 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.851147 (+ 7290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:14.851172 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:14.851200 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:14.851250 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:14.856011 (+ 4761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.857596 (+ 1585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.857608 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.857613 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.857798 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.858332 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.858339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.858343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.858953 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:14.859429 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.859817 (+ 388us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.860086 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349}
W20260929 16:04:14.861531 22299 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55770
I20260929 16:04:14.897732 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.865829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56982 (local address 127.20.121.190:40505)
0929 16:04:14.878070 (+ 12241us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.878078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.878739 (+ 661us) server_negotiation.cc:407] Connection header received
0929 16:04:14.879061 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.879069 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.879165 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.879254 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:14.881446 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.882889 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.884893 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.885628 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.885771 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.889696 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:14.889704 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:14.889710 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:14.889760 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:14.893764 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.894759 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.894768 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.894773 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.894943 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.895425 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.895432 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.895437 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.896148 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:14.896429 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.896920 (+ 491us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.897297 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12035,"thread_start_us":12255,"threads_started":1}
W20260929 16:04:14.898427 22097 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56982
I20260929 16:04:14.900952 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22169
I20260929 16:04:14.921509 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.903077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55786 (local address 127.20.121.129:40013)
0929 16:04:14.903715 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.903722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.904208 (+ 486us) server_negotiation.cc:407] Connection header received
0929 16:04:14.904390 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.904396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.904490 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.904610 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:14.906249 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.907478 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.909372 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.910221 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.910357 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.913682 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:14.913690 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:14.913695 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:14.913742 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:14.917906 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.919200 (+ 1294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.919207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.919211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.919396 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.919844 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.919849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.919853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.920216 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:14.920491 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.920817 (+ 326us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.921095 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505}
W20260929 16:04:14.922304 22299 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55786
W20260929 16:04:14.922925 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22169: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:04:14.923117 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 22169
I20260929 16:04:14.939942 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22309
Sep 29 16:04:14 dist-test-slave-x07d krb5kdc[22053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697853, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:04:14.958693 22599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.940899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.20.121.130:35529)
0929 16:04:14.941101 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.941107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.941128 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:04:14.941291 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.941297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.941389 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.941519 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:14.942890 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.944029 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.945428 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.946211 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.946355 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.951243 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:14.951264 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:14.951286 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:14.951339 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:14.955408 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.956265 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.956277 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.956281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.956475 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.956851 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.956856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.956859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.957330 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:14.957674 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:14.957987 (+ 313us) server_negotiation.cc:299] Negotiation successful
0929 16:04:14.958274 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260929 16:04:14.959470 22439 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53802
W20260929 16:04:14.960235 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22309: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:04:14.960445 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 22309
I20260929 16:04:14.982743 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22448
Sep 29 16:04:14 dist-test-slave-x07d krb5kdc[22053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697853, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:04:15.001696 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:14.983607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.20.121.131:37907)
0929 16:04:14.983828 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:14.983833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:14.983855 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:04:14.984069 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:14.984073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:14.984145 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:14.984222 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:14.985906 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.986873 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.988743 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:14.989232 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:14.989338 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:14.994935 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:14.994951 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:14.994968 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:14.995013 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:14.998530 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.999298 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.999310 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.999312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:14.999451 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:14.999898 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:14.999903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:14.999911 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:15.000312 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:15.000648 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:15.000981 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 16:04:15.001221 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
W20260929 16:04:15.002485 22578 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57664
W20260929 16:04:15.003188 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22448: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:04:15.003346 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 22448
I20260929 16:04:15.023183 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22066
I20260929 16:04:15.024327 22138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:15.145522 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 22066
2026-09-29T16:04:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4943 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:15 dist-test-slave-x07d krb5kdc[22678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:15 dist-test-slave-x07d krb5kdc[22678](info): set up 2 sockets
Sep 29 16:04:15 dist-test-slave-x07d krb5kdc[22678](info): commencing operation
krb5kdc: starting...
W20260929 16:04:16.314751 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:16 dist-test-slave-x07d krb5kdc[22678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:04:16.513244 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:32843
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:32843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:16.845196 22691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:16.845494 22691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:16.845616 22691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:16.852941 22691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:16.853060 22691 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 16:04:16.853106 22691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:16.853163 22691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:16.853210 22691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:16.853271 22691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:16.863385 22691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:32843
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:32843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:16.864650 22691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:16.865922 22691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:16.878315 22696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:16.878247 22699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:16.878284 22697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:16.879189 22691 server_base.cc:1034] running on GCE node
I20260929 16:04:16.879947 22691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:16.881423 22691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:16.882663 22691 hybrid_clock.cc:648] HybridClock initialized: now 1790697856882598 us; error 43 us; skew 500 ppm
I20260929 16:04:16.883070 22691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:16 dist-test-slave-x07d krb5kdc[22678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697856, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:16.886449 22691 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:04:16.888080 22691 webserver.cc:460] Webserver started at http://127.20.121.190:41197/ using document root <none> and password file <none>
I20260929 16:04:16.888825 22691 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:16.888958 22691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:16.889282 22691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:16.892501 22691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "1fa2cefd26594a98a263fa38903325be"
format_stamp: "Formatted at 2026-09-29 16:04:16 on dist-test-slave-x07d"
server_key: "477ecd341fb43ea824053534d02b997c"
server_key_iv: "862275892bee0806cf65c800802320df"
server_key_version: "encryptionkey@0"
I20260929 16:04:16.893316 22691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "1fa2cefd26594a98a263fa38903325be"
format_stamp: "Formatted at 2026-09-29 16:04:16 on dist-test-slave-x07d"
server_key: "477ecd341fb43ea824053534d02b997c"
server_key_iv: "862275892bee0806cf65c800802320df"
server_key_version: "encryptionkey@0"
I20260929 16:04:16.899179 22691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260929 16:04:16.903138 22706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:16.904798 22691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 16:04:16.905074 22691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "1fa2cefd26594a98a263fa38903325be"
format_stamp: "Formatted at 2026-09-29 16:04:16 on dist-test-slave-x07d"
server_key: "477ecd341fb43ea824053534d02b997c"
server_key_iv: "862275892bee0806cf65c800802320df"
server_key_version: "encryptionkey@0"
I20260929 16:04:16.905334 22691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:16.928648 22691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:16.933300 22691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:16.933621 22691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:16.956862 22691 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:32843
I20260929 16:04:16.956885 22767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:32843 every 8 connection(s)
I20260929 16:04:16.958384 22691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:16.962468 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 22691
I20260929 16:04:16.962504 22768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:16.962671 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:16.963325 20966 external_mini_cluster.cc:1428] Setting key 6d54e71e359e14820e2f1f1efa01b356
I20260929 16:04:16.977867 22768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be: Bootstrap starting.
Sep 29 16:04:16 dist-test-slave-x07d krb5kdc[22678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:16.981757 22768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:16.983407 22768 log.cc:824] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:16.987102 22768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be: No bootstrap required, opened a new log
I20260929 16:04:16.990374 22771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:16.965499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.20.121.190:32843)
0929 16:04:16.966367 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:16.966380 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:16.966423 (+ 43us) server_negotiation.cc:407] Connection header received
0929 16:04:16.968864 (+ 2441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:16.968890 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:16.969333 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:16.969712 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:16.971763 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:16.973155 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:16.975245 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:16.975991 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:16.976160 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:16.981070 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:16.981118 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:16.981134 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:16.981174 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:16.985221 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:16.986048 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:16.986058 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:16.986067 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:16.986301 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:16.986715 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:16.986720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:16.986724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:16.987468 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:16.987803 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:16.988219 (+ 416us) server_negotiation.cc:299] Negotiation successful
0929 16:04:16.988846 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":414,"threads_started":1}
I20260929 16:04:16.996507 22768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa2cefd26594a98a263fa38903325be" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 32843 } }
I20260929 16:04:16.996866 22768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:16.996976 22768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa2cefd26594a98a263fa38903325be, State: Initialized, Role: FOLLOWER
I20260929 16:04:16.997663 22768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa2cefd26594a98a263fa38903325be" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 32843 } }
I20260929 16:04:16.997920 22768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:16.998082 22768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:16.998265 22768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:17.000474 22768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa2cefd26594a98a263fa38903325be" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 32843 } }
I20260929 16:04:17.001026 22768 leader_election.cc:302] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fa2cefd26594a98a263fa38903325be; no voters:
I20260929 16:04:17.001693 22768 leader_election.cc:288] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:17.001902 22773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:17.002859 22773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [term 1 LEADER]: Becoming Leader. State: Replica: 1fa2cefd26594a98a263fa38903325be, State: Running, Role: LEADER
I20260929 16:04:17.003429 22773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa2cefd26594a98a263fa38903325be" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 32843 } }
I20260929 16:04:17.004107 22768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:17.006613 22775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fa2cefd26594a98a263fa38903325be. Latest consensus state: current_term: 1 leader_uuid: "1fa2cefd26594a98a263fa38903325be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa2cefd26594a98a263fa38903325be" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 32843 } } }
I20260929 16:04:17.006991 22775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:17.006536 22774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fa2cefd26594a98a263fa38903325be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fa2cefd26594a98a263fa38903325be" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 32843 } } }
I20260929 16:04:17.007388 22774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:17.008673 22780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:17.016182 22780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:17.025674 22780 catalog_manager.cc:1347] Generated new cluster ID: 027416b540884759abde9cb38368c434
I20260929 16:04:17.025818 22780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:17.037631 22780 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:17.039053 22780 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:17.052531 22780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be: Generated new TSK 0
I20260929 16:04:17.053455 22780 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:04:17.109961 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:32843
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:17.454520 22794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:17.454923 22794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:17.455114 22794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:17.462761 22794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:17.462960 22794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:17.463126 22794 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 16:04:17.463217 22794 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 16:04:17.463335 22794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:17.473717 22794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.121.190:32843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:17.475198 22794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:17.476646 22794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:17.491922 22800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:17.492393 22802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:17.492528 22799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:17.493232 22794 server_base.cc:1034] running on GCE node
I20260929 16:04:17.493933 22794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:17.495151 22794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:17.496436 22794 hybrid_clock.cc:648] HybridClock initialized: now 1790697857496357 us; error 34 us; skew 500 ppm
I20260929 16:04:17.496825 22794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:17 dist-test-slave-x07d krb5kdc[22678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697857, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:17.500654 22794 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:04:17.502415 22794 webserver.cc:460] Webserver started at http://127.20.121.129:37423/ using document root <none> and password file <none>
I20260929 16:04:17.503157 22794 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:17.503266 22794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:17.503549 22794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:17.507149 22794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "3e8215f6e43b48ebb8e66a779f71f3fd"
format_stamp: "Formatted at 2026-09-29 16:04:17 on dist-test-slave-x07d"
server_key: "3143a33c6b72d4ab4c143d6dd8e13878"
server_key_iv: "f27f67c7864dc43467e7bbafd9ddcb2b"
server_key_version: "encryptionkey@0"
I20260929 16:04:17.507921 22794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "3e8215f6e43b48ebb8e66a779f71f3fd"
format_stamp: "Formatted at 2026-09-29 16:04:17 on dist-test-slave-x07d"
server_key: "3143a33c6b72d4ab4c143d6dd8e13878"
server_key_iv: "f27f67c7864dc43467e7bbafd9ddcb2b"
server_key_version: "encryptionkey@0"
I20260929 16:04:17.514245 22794 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 16:04:17.518045 22809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:17.519801 22794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:04:17.520005 22794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "3e8215f6e43b48ebb8e66a779f71f3fd"
format_stamp: "Formatted at 2026-09-29 16:04:17 on dist-test-slave-x07d"
server_key: "3143a33c6b72d4ab4c143d6dd8e13878"
server_key_iv: "f27f67c7864dc43467e7bbafd9ddcb2b"
server_key_version: "encryptionkey@0"
I20260929 16:04:17.520238 22794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:17.539840 22794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:17.544215 22794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:17.544534 22794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:17.547428 22794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:17.547557 22794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:17.547709 22794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:17.547809 22794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:04:17 dist-test-slave-x07d krb5kdc[22678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697857, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:17.581007 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:17.547963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:57827 (local address 127.20.121.190:32843)
0929 16:04:17.548747 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:17.548752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:17.549695 (+ 943us) server_negotiation.cc:407] Connection header received
0929 16:04:17.552014 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:17.552024 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:17.552185 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:17.552319 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:17.555395 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:17.556378 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:17.559943 (+ 3565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:17.560523 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:17.560674 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:17.566535 (+ 5861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:17.566541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:17.566546 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:17.566588 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:17.576873 (+ 10285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:17.577807 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:17.577816 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:17.577823 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:17.578064 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:17.578750 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:17.578756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:17.578759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:17.579157 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:17.579345 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:17.580359 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0929 16:04:17.580629 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":579,"threads_started":1}
I20260929 16:04:17.582760 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:17.548683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:32843 (local address 127.20.121.129:57827)
0929 16:04:17.549519 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:17.549560 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:17.551704 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:17.552651 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:17.552668 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:17.553720 (+ 1052us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:17.555050 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:17.555096 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:17.556624 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:17.556632 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:17.559705 (+ 3073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:17.559734 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:17.560825 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:17.560839 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:17.561193 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:17.562166 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:17.562196 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:17.566092 (+ 3896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:17.577094 (+ 11002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:17.577103 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:17.577120 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:17.577638 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:17.578217 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:17.578226 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:17.578232 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:17.578552 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:17.579357 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:17.579365 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:17.579868 (+ 503us) client_negotiation.cc:769] Sending connection context
0929 16:04:17.580238 (+ 370us) client_negotiation.cc:240] Negotiation successful
0929 16:04:17.580871 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":361,"threads_started":1}
I20260929 16:04:17.592996 22794 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:40363
I20260929 16:04:17.593067 22929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:40363 every 8 connection(s)
I20260929 16:04:17.594939 22794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:17.597411 22930 heartbeater.cc:347] Connected to a master server at 127.20.121.190:32843
I20260929 16:04:17.597741 22930 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:17.598789 22930 heartbeater.cc:511] Master 127.20.121.190:32843 requested a full tablet report, sending...
I20260929 16:04:17.601850 22723 ts_manager.cc:194] Registered new tserver with Master: 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363)
I20260929 16:04:17.602425 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 22794
I20260929 16:04:17.602646 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:17.603267 20966 external_mini_cluster.cc:1428] Setting key 1b6989164158fe81663e1747f2cb1252
I20260929 16:04:17.604460 22723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:57827
I20260929 16:04:17.607429 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:17.590872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:39411 (local address 127.20.121.190:32843)
0929 16:04:17.591054 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:17.591059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:17.591668 (+ 609us) server_negotiation.cc:407] Connection header received
0929 16:04:17.591925 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:17.591932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:17.592048 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:17.592187 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:17.593511 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:17.594652 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:17.596660 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:17.597376 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:17.597489 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:17.600046 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:17.600055 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:17.600061 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:17.600110 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:17.604047 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:17.604962 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:17.604971 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:17.604974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:17.605131 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:17.605638 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:17.605643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:17.605647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:17.606102 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:17.606308 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:17.606755 (+ 447us) server_negotiation.cc:299] Negotiation successful
0929 16:04:17.607014 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 16:04:17.607573 22925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:17.590905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:32843 (local address 127.20.121.129:39411)
0929 16:04:17.591537 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:17.591559 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:17.591754 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:17.592232 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:17.592238 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:17.592881 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:17.593330 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:17.593347 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:17.594836 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:17.594841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:17.596447 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:17.596465 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:17.597584 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:17.597592 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:17.597800 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:17.598458 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:17.598484 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:17.599644 (+ 1160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:17.604253 (+ 4609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:17.604259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:17.604262 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:17.604801 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:17.605280 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:17.605285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:17.605291 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:17.605479 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:17.606247 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:17.606253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:17.606468 (+ 215us) client_negotiation.cc:769] Sending connection context
0929 16:04:17.606646 (+ 178us) client_negotiation.cc:240] Negotiation successful
0929 16:04:17.607022 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":338,"threads_started":1}
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:04:17.651506 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:32843
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:17.990577 22934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:17.990895 22934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:17.991111 22934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:17.998790 22934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:17.998917 22934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:17.999065 22934 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 16:04:17.999125 22934 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 16:04:17.999256 22934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:18.010416 22934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.121.190:32843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:18.012189 22934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:18.013700 22934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:18.029179 22940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:18.029043 22939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:18.029072 22942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:18.029958 22934 server_base.cc:1034] running on GCE node
I20260929 16:04:18.030665 22934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:18.031654 22934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:18.032871 22934 hybrid_clock.cc:648] HybridClock initialized: now 1790697858032811 us; error 47 us; skew 500 ppm
I20260929 16:04:18.033281 22934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:18 dist-test-slave-x07d krb5kdc[22678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697858, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:18.036579 22934 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:04:18.038288 22934 webserver.cc:460] Webserver started at http://127.20.121.130:43301/ using document root <none> and password file <none>
I20260929 16:04:18.039042 22934 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:18.039191 22934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:18.039505 22934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:18.042668 22934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "95a95386def14a4fb180faf7d5132622"
format_stamp: "Formatted at 2026-09-29 16:04:18 on dist-test-slave-x07d"
server_key: "e96b1dd1854e4c0ac8df7e322c18c8ad"
server_key_iv: "b721b6c9973b5d85647a2cbe2e735a93"
server_key_version: "encryptionkey@0"
I20260929 16:04:18.043423 22934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "95a95386def14a4fb180faf7d5132622"
format_stamp: "Formatted at 2026-09-29 16:04:18 on dist-test-slave-x07d"
server_key: "e96b1dd1854e4c0ac8df7e322c18c8ad"
server_key_iv: "b721b6c9973b5d85647a2cbe2e735a93"
server_key_version: "encryptionkey@0"
I20260929 16:04:18.049208 22934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 16:04:18.052754 22949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:18.054250 22934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:18.054415 22934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "95a95386def14a4fb180faf7d5132622"
format_stamp: "Formatted at 2026-09-29 16:04:18 on dist-test-slave-x07d"
server_key: "e96b1dd1854e4c0ac8df7e322c18c8ad"
server_key_iv: "b721b6c9973b5d85647a2cbe2e735a93"
server_key_version: "encryptionkey@0"
I20260929 16:04:18.054615 22934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:18.071779 22934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:18.076484 22934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:18.076824 22934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:18.080035 22934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:18.080150 22934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:18.080281 22934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:18.080358 22934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:04:18 dist-test-slave-x07d krb5kdc[22678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697858, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:18.108716 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.081025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:59491 (local address 127.20.121.190:32843)
0929 16:04:18.081244 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:18.081251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:18.083047 (+ 1796us) server_negotiation.cc:407] Connection header received
0929 16:04:18.085974 (+ 2927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:18.085982 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:18.086131 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:18.086247 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:18.089223 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.090367 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.092988 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.093609 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.093777 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.099775 (+ 5998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:18.099783 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:18.099790 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:18.099845 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:18.103848 (+ 4003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:18.105147 (+ 1299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:18.105157 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:18.105162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:18.105364 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:18.106417 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:18.106425 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:18.106430 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:18.106853 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:18.107026 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:18.108090 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0929 16:04:18.108340 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:04:18.110621 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.081599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:32843 (local address 127.20.121.130:59491)
0929 16:04:18.082813 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:18.082870 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:18.085472 (+ 2602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:18.086967 (+ 1495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:18.086981 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:18.087829 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:18.088967 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.089002 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.090566 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.090597 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.092801 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.092816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.094162 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.094167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.094487 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.095500 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:18.095550 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:18.099471 (+ 3921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:18.104091 (+ 4620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:18.104105 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:18.104136 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:18.104955 (+ 819us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:18.105561 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:18.105570 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:18.105577 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:18.106237 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:18.107014 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:18.107023 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:18.107611 (+ 588us) client_negotiation.cc:769] Sending connection context
0929 16:04:18.107983 (+ 372us) client_negotiation.cc:240] Negotiation successful
0929 16:04:18.108633 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":439,"threads_started":1}
I20260929 16:04:18.128916 22934 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:39419
I20260929 16:04:18.128960 23068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:39419 every 8 connection(s)
I20260929 16:04:18.130524 22934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:18.131191 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 22934
I20260929 16:04:18.131386 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:18.132135 20966 external_mini_cluster.cc:1428] Setting key c34137fbaf646620e2f554180632e287
I20260929 16:04:18.133109 23069 heartbeater.cc:347] Connected to a master server at 127.20.121.190:32843
I20260929 16:04:18.133424 23069 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:18.134497 23069 heartbeater.cc:511] Master 127.20.121.190:32843 requested a full tablet report, sending...
I20260929 16:04:18.135989 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.118510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:32843 (local address 127.20.121.130:36353)
0929 16:04:18.119322 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:18.119349 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:18.119559 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:18.119930 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:18.119938 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:18.120697 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:18.121197 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.121216 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.122557 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.122570 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.124957 (+ 2387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.124990 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.125879 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.125889 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.126198 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.127166 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:18.127185 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:18.128080 (+ 895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:18.132543 (+ 4463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:18.132548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:18.132552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:18.133166 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:18.133691 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:18.133696 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:18.133699 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:18.133852 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:18.134723 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:18.134729 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:18.134969 (+ 240us) client_negotiation.cc:769] Sending connection context
0929 16:04:18.135125 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 16:04:18.135500 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":396,"threads_started":1}
I20260929 16:04:18.136111 22723 ts_manager.cc:194] Registered new tserver with Master: 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419)
I20260929 16:04:18.136989 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.118559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:36353 (local address 127.20.121.190:32843)
0929 16:04:18.118814 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:18.118824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:18.119482 (+ 658us) server_negotiation.cc:407] Connection header received
0929 16:04:18.119705 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:18.119709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:18.119775 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:18.119860 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:18.121438 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.122374 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.125213 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.125634 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.125752 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.128414 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:18.128425 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:18.128433 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:18.128497 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:18.132354 (+ 3857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:18.133337 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:18.133344 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:18.133350 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:18.133550 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:18.134044 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:18.134049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:18.134052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:18.134522 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:18.134664 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:18.136411 (+ 1747us) server_negotiation.cc:299] Negotiation successful
0929 16:04:18.136642 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 16:04:18.137223 22723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:59491
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:04:18.182806 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:32843
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:18.521104 23073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:18.521478 23073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:18.521672 23073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:18.529326 23073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:18.529515 23073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:18.529675 23073 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 16:04:18.529771 23073 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 16:04:18.529911 23073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:18.540385 23073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.121.190:32843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:18.541875 23073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:18.543313 23073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:18.557813 23081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:18.557888 23079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:18.557909 23078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:18.559300 23073 server_base.cc:1034] running on GCE node
I20260929 16:04:18.560097 23073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:18.561311 23073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:18.562646 23073 hybrid_clock.cc:648] HybridClock initialized: now 1790697858562549 us; error 60 us; skew 500 ppm
I20260929 16:04:18.563054 23073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:18 dist-test-slave-x07d krb5kdc[22678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697858, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:18.566711 23073 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:04:18.568497 23073 webserver.cc:460] Webserver started at http://127.20.121.131:41549/ using document root <none> and password file <none>
I20260929 16:04:18.569242 23073 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:18.569355 23073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:18.569636 23073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:18.572873 23073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "d537755f82cf42f4898789f370d67c3a"
format_stamp: "Formatted at 2026-09-29 16:04:18 on dist-test-slave-x07d"
server_key: "da8e87876869d559e82b91a73a010631"
server_key_iv: "da1be169c7ba20ce7632c52f149563d6"
server_key_version: "encryptionkey@0"
I20260929 16:04:18.573658 23073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "d537755f82cf42f4898789f370d67c3a"
format_stamp: "Formatted at 2026-09-29 16:04:18 on dist-test-slave-x07d"
server_key: "da8e87876869d559e82b91a73a010631"
server_key_iv: "da1be169c7ba20ce7632c52f149563d6"
server_key_version: "encryptionkey@0"
I20260929 16:04:18.579728 23073 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260929 16:04:18.584081 23088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:18.585922 23073 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260929 16:04:18.586196 23073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "d537755f82cf42f4898789f370d67c3a"
format_stamp: "Formatted at 2026-09-29 16:04:18 on dist-test-slave-x07d"
server_key: "da8e87876869d559e82b91a73a010631"
server_key_iv: "da1be169c7ba20ce7632c52f149563d6"
server_key_version: "encryptionkey@0"
I20260929 16:04:18.586417 23073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:18.601480 23073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:18.606664 23073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:18.607026 23073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:18.610534 22930 heartbeater.cc:503] Master 127.20.121.190:32843 was elected leader, sending a full tablet report...
I20260929 16:04:18.614640 23073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:18.614794 23073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:18.614919 23073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:18.615013 23073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:04:18 dist-test-slave-x07d krb5kdc[22678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697858, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:18.642040 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.614648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:59715 (local address 127.20.121.190:32843)
0929 16:04:18.614899 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:18.614907 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:18.616509 (+ 1602us) server_negotiation.cc:407] Connection header received
0929 16:04:18.620120 (+ 3611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:18.620125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:18.620208 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:18.620307 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:18.623517 (+ 3210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.624939 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.627565 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.628536 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.628734 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.633947 (+ 5213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:18.633972 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:18.633979 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:18.634086 (+ 107us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:18.637528 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:18.638985 (+ 1457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:18.638994 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:18.638999 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:18.639205 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:18.639822 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:18.639829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:18.639832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:18.640141 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:18.640308 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:18.641299 (+ 991us) server_negotiation.cc:299] Negotiation successful
0929 16:04:18.641583 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 16:04:18.643169 23096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.613218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:32843 (local address 127.20.121.131:59715)
0929 16:04:18.616239 (+ 3021us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:18.616323 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:18.619654 (+ 3331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:18.620693 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:18.620708 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:18.621902 (+ 1194us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:18.623155 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.623190 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.625298 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.625304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.627328 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.627346 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.628854 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.628860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.629067 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.630070 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:18.630112 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:18.633571 (+ 3459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:18.637751 (+ 4180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:18.637768 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:18.637798 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:18.638772 (+ 974us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:18.639357 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:18.639367 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:18.639371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:18.639664 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:18.640285 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:18.640295 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:18.640815 (+ 520us) client_negotiation.cc:769] Sending connection context
0929 16:04:18.641147 (+ 332us) client_negotiation.cc:240] Negotiation successful
0929 16:04:18.641815 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2546,"thread_start_us":2118,"threads_started":1}
I20260929 16:04:18.668282 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.650834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:45155 (local address 127.20.121.190:32843)
0929 16:04:18.651061 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:18.651070 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:18.651900 (+ 830us) server_negotiation.cc:407] Connection header received
0929 16:04:18.652141 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:18.652146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:18.652210 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:18.652296 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:18.653820 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.654915 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.657579 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.658353 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.658497 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.661499 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:18.661506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:18.661513 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:18.661563 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:18.665073 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:18.666182 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:18.666187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:18.666191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:18.666342 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:18.666738 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:18.666744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:18.666747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:18.667038 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:18.667209 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:18.667659 (+ 450us) server_negotiation.cc:299] Negotiation successful
0929 16:04:18.667912 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 16:04:18.668380 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.650920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:32843 (local address 127.20.121.131:45155)
0929 16:04:18.651746 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:18.651772 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:18.651972 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:18.652382 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:18.652390 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:18.653134 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:18.653626 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.653643 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.655093 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.655101 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.657351 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.657375 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.658605 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.658612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.658796 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.659610 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:18.659650 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:18.661220 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:18.665302 (+ 4082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:18.665308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:18.665312 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:18.666000 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:18.666477 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:18.666482 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:18.666486 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:18.666624 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:18.667179 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:18.667183 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:18.667381 (+ 198us) client_negotiation.cc:769] Sending connection context
0929 16:04:18.667556 (+ 175us) client_negotiation.cc:240] Negotiation successful
0929 16:04:18.667936 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":402,"threads_started":1}
I20260929 16:04:18.682890 23073 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:34383
I20260929 16:04:18.682906 23207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:34383 every 8 connection(s)
I20260929 16:04:18.684311 23073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:18.684885 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 23073
I20260929 16:04:18.685082 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:18.685736 20966 external_mini_cluster.cc:1428] Setting key f0a4adad4243ff73c201bb8d102b2c1b
I20260929 16:04:18.686978 23208 heartbeater.cc:347] Connected to a master server at 127.20.121.190:32843
I20260929 16:04:18.687265 23208 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:18.688038 23208 heartbeater.cc:511] Master 127.20.121.190:32843 requested a full tablet report, sending...
I20260929 16:04:18.689620 22722 ts_manager.cc:194] Registered new tserver with Master: d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383)
I20260929 16:04:18.690608 22722 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:59715
I20260929 16:04:18.701543 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 16:04:18 dist-test-slave-x07d krb5kdc[22678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 16:04:18 dist-test-slave-x07d krb5kdc[22678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:18.751112 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.736462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.20.121.190:32843)
0929 16:04:18.736715 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:18.736724 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:18.736858 (+ 134us) server_negotiation.cc:407] Connection header received
0929 16:04:18.737081 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:18.737086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:18.737161 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:18.737260 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:18.738816 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.739719 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.741025 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.741472 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.741586 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.745591 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:18.745597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:18.745600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:18.745640 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:18.748372 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:18.749057 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:18.749069 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:18.749072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:18.749198 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:18.749579 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:18.749586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:18.749589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:18.749866 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:18.750159 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:18.750474 (+ 315us) server_negotiation.cc:299] Negotiation successful
0929 16:04:18.750718 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260929 16:04:18.756244 22722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:04:18.763635 22722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:18.791316 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.778236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.20.121.130:39419)
0929 16:04:18.778961 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:18.778971 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:18.779922 (+ 951us) server_negotiation.cc:407] Connection header received
0929 16:04:18.780226 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:18.780236 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:18.780552 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:18.780764 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:18.782071 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.784031 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.787524 (+ 3493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.789524 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.789696 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.789906 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:18.790643 (+ 737us) server_negotiation.cc:299] Negotiation successful
0929 16:04:18.790895 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":439,"threads_started":1}
I20260929 16:04:18.792485 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.777513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39419 (local address 127.0.0.1:49320)
0929 16:04:18.779798 (+ 2285us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:18.779825 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:18.780031 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:18.781346 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:18.781351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:18.781379 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:18.781849 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.781860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.784279 (+ 2419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.784287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.787259 (+ 2972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.787284 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.790116 (+ 2832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.790122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.790344 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.790409 (+ 65us) client_negotiation.cc:769] Sending connection context
0929 16:04:18.791691 (+ 1282us) client_negotiation.cc:240] Negotiation successful
0929 16:04:18.791909 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2159,"mutex_wait_us":28,"thread_start_us":1972,"threads_started":1}
I20260929 16:04:18.793550 23221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.777482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40363 (local address 127.0.0.1:51188)
0929 16:04:18.778335 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:18.778384 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:18.778558 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:18.780487 (+ 1929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:18.780499 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:18.780541 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:18.781040 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.781127 (+ 87us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.783900 (+ 2773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.783909 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.787647 (+ 3738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.787669 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.790697 (+ 3028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.790708 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.791059 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.791116 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 16:04:18.791228 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 16:04:18.792949 (+ 1721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"mutex_wait_us":77,"thread_start_us":411,"threads_started":1}
I20260929 16:04:18.792519 23223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.777845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.20.121.129:40363)
0929 16:04:18.779404 (+ 1559us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:18.779416 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:18.779447 (+ 31us) server_negotiation.cc:407] Connection header received
0929 16:04:18.779550 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:18.779559 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:18.779910 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:18.780178 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:18.781398 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.783630 (+ 2232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.787930 (+ 4300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.790083 (+ 2153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.790316 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.790532 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:18.791812 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0929 16:04:18.792087 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1378,"thread_start_us":1256,"threads_started":1}
I20260929 16:04:18.796406 22999 tablet_service.cc:1467] Processing CreateTablet for tablet d19ff3e5fc514243914374d33106a7e7 (DEFAULT_TABLE table=test-table [id=8e0d6c15e2be472194d772ab1ccf0f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:18.797665 22999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d19ff3e5fc514243914374d33106a7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:18.802273 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.779538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:34383 (local address 127.0.0.1:36110)
0929 16:04:18.792759 (+ 13221us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:18.792780 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:18.792888 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:18.795545 (+ 2657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:18.795550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:18.795580 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:18.795775 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.795786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.798521 (+ 2735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.798526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.800128 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:18.800148 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.801443 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.801452 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:18.801636 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.801668 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 16:04:18.801769 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 16:04:18.801910 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13122,"thread_start_us":18886,"threads_started":1}
I20260929 16:04:18.802316 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:18.780078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.20.121.131:34383)
0929 16:04:18.794723 (+ 14645us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:18.794736 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:18.794781 (+ 45us) server_negotiation.cc:407] Connection header received
0929 16:04:18.794910 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:18.794923 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:18.795291 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:18.795548 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:18.795948 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.798321 (+ 2373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.800313 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:18.801268 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:18.801486 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:18.801683 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:18.801828 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 16:04:18.801984 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14392,"thread_start_us":17999,"threads_started":1}
I20260929 16:04:18.802563 22860 tablet_service.cc:1467] Processing CreateTablet for tablet d19ff3e5fc514243914374d33106a7e7 (DEFAULT_TABLE table=test-table [id=8e0d6c15e2be472194d772ab1ccf0f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:18.804224 22860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d19ff3e5fc514243914374d33106a7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:18.805788 23143 tablet_service.cc:1467] Processing CreateTablet for tablet d19ff3e5fc514243914374d33106a7e7 (DEFAULT_TABLE table=test-table [id=8e0d6c15e2be472194d772ab1ccf0f01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:18.807137 23143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d19ff3e5fc514243914374d33106a7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:18.814080 23227 tablet_bootstrap.cc:492] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: Bootstrap starting.
I20260929 16:04:18.818126 23227 tablet_bootstrap.cc:654] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:18.818675 23228 tablet_bootstrap.cc:492] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Bootstrap starting.
I20260929 16:04:18.819744 23227 log.cc:824] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:18.821792 23230 tablet_bootstrap.cc:492] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: Bootstrap starting.
I20260929 16:04:18.822503 23228 tablet_bootstrap.cc:654] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:18.823827 23228 log.cc:824] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:18.824157 23227 tablet_bootstrap.cc:492] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: No bootstrap required, opened a new log
I20260929 16:04:18.824481 23227 ts_tablet_manager.cc:1397] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260929 16:04:18.825568 23230 tablet_bootstrap.cc:654] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:18.826856 23230 log.cc:824] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:18.827096 23228 tablet_bootstrap.cc:492] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: No bootstrap required, opened a new log
I20260929 16:04:18.827368 23228 ts_tablet_manager.cc:1397] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260929 16:04:18.829854 23230 tablet_bootstrap.cc:492] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: No bootstrap required, opened a new log
I20260929 16:04:18.830153 23230 ts_tablet_manager.cc:1397] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260929 16:04:18.833140 23227 raft_consensus.cc:348] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:18.833588 23227 raft_consensus.cc:374] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:18.833690 23227 raft_consensus.cc:729] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a95386def14a4fb180faf7d5132622, State: Initialized, Role: FOLLOWER
I20260929 16:04:18.834407 23227 consensus_queue.cc:260] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:18.835552 23069 heartbeater.cc:503] Master 127.20.121.190:32843 was elected leader, sending a full tablet report...
I20260929 16:04:18.836294 23227 ts_tablet_manager.cc:1428] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: Time spent starting tablet: real 0.012s user 0.009s sys 0.002s
I20260929 16:04:18.838187 23228 raft_consensus.cc:348] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:18.838641 23228 raft_consensus.cc:374] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:18.838739 23228 raft_consensus.cc:729] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8215f6e43b48ebb8e66a779f71f3fd, State: Initialized, Role: FOLLOWER
I20260929 16:04:18.839402 23228 consensus_queue.cc:260] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:18.841298 23228 ts_tablet_manager.cc:1428] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260929 16:04:18.841174 23230 raft_consensus.cc:348] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:18.841634 23230 raft_consensus.cc:374] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:18.841749 23230 raft_consensus.cc:729] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d537755f82cf42f4898789f370d67c3a, State: Initialized, Role: FOLLOWER
I20260929 16:04:18.842474 23230 consensus_queue.cc:260] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:18.843858 23208 heartbeater.cc:503] Master 127.20.121.190:32843 was elected leader, sending a full tablet report...
I20260929 16:04:18.844415 23230 ts_tablet_manager.cc:1428] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260929 16:04:18.847658 22931 tablet.cc:2369] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:04:18.893646 23070 tablet.cc:2369] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:04:18.936161 23209 tablet.cc:2369] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:04:19.010450 23233 raft_consensus.cc:482] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:19.010977 23233 raft_consensus.cc:504] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.012873 23233 leader_election.cc:288] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363), d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383)
I20260929 16:04:19.020010 23223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.013342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:47069 (local address 127.20.121.129:40363)
0929 16:04:19.013599 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.013609 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.013984 (+ 375us) server_negotiation.cc:407] Connection header received
0929 16:04:19.014281 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.014287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.014366 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.014435 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:19.015288 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.016000 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.017819 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.018963 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.019130 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.019204 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.019605 (+ 401us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.019754 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 16:04:19.020035 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.013108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40363 (local address 127.20.121.130:47069)
0929 16:04:19.013839 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:19.013869 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:19.014085 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:19.014658 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:19.014665 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:19.014712 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:19.015144 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.015155 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.016143 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.016151 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.017654 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.017666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.019189 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.019196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.019414 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.019462 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 16:04:19.019564 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 16:04:19.019741 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":328,"threads_started":1}
I20260929 16:04:19.021085 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.014097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:60907 (local address 127.20.121.131:34383)
0929 16:04:19.014343 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.014350 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.014615 (+ 265us) server_negotiation.cc:407] Connection header received
0929 16:04:19.014744 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.014752 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.014844 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.014940 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:19.015538 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.016274 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.018513 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.019767 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.019969 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.020090 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.020622 (+ 532us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.020782 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260929 16:04:19.021163 23237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.013858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:34383 (local address 127.20.121.130:60907)
0929 16:04:19.014512 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:19.014526 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:19.014629 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:19.014959 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:19.014962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:19.014979 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:19.015358 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.015365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.016615 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.016623 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.018325 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.018343 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.020320 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.020327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.020461 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.020483 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 16:04:19.020556 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 16:04:19.020724 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":1,"thread_start_us":350,"threads_started":1}
I20260929 16:04:19.021566 22880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d19ff3e5fc514243914374d33106a7e7" candidate_uuid: "95a95386def14a4fb180faf7d5132622" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" is_pre_election: true
I20260929 16:04:19.022248 22880 raft_consensus.cc:2454] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95a95386def14a4fb180faf7d5132622 in term 0.
I20260929 16:04:19.022420 23163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d19ff3e5fc514243914374d33106a7e7" candidate_uuid: "95a95386def14a4fb180faf7d5132622" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d537755f82cf42f4898789f370d67c3a" is_pre_election: true
I20260929 16:04:19.023011 22950 leader_election.cc:302] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8215f6e43b48ebb8e66a779f71f3fd, 95a95386def14a4fb180faf7d5132622; no voters:
I20260929 16:04:19.022997 23163 raft_consensus.cc:2454] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95a95386def14a4fb180faf7d5132622 in term 0.
I20260929 16:04:19.023450 23233 raft_consensus.cc:2781] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:19.023661 23233 raft_consensus.cc:482] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:19.023794 23233 raft_consensus.cc:3037] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:19.027046 23233 raft_consensus.cc:504] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.027697 23233 leader_election.cc:288] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [CANDIDATE]: Term 1 election: Requested vote from peers 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363), d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383)
I20260929 16:04:19.028292 22880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d19ff3e5fc514243914374d33106a7e7" candidate_uuid: "95a95386def14a4fb180faf7d5132622" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd"
I20260929 16:04:19.028329 23163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d19ff3e5fc514243914374d33106a7e7" candidate_uuid: "95a95386def14a4fb180faf7d5132622" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d537755f82cf42f4898789f370d67c3a"
I20260929 16:04:19.028510 22880 raft_consensus.cc:3037] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:19.028537 23163 raft_consensus.cc:3037] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:19.030871 22880 raft_consensus.cc:2454] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95a95386def14a4fb180faf7d5132622 in term 1.
I20260929 16:04:19.031068 23163 raft_consensus.cc:2454] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95a95386def14a4fb180faf7d5132622 in term 1.
I20260929 16:04:19.031399 22950 leader_election.cc:302] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8215f6e43b48ebb8e66a779f71f3fd, 95a95386def14a4fb180faf7d5132622; no voters:
I20260929 16:04:19.031810 23233 raft_consensus.cc:2781] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:19.032351 23233 raft_consensus.cc:686] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 1 LEADER]: Becoming Leader. State: Replica: 95a95386def14a4fb180faf7d5132622, State: Running, Role: LEADER
I20260929 16:04:19.032903 23233 consensus_queue.cc:237] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.037601 22721 catalog_manager.cc:5495] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95a95386def14a4fb180faf7d5132622 (127.20.121.130). New cstate: current_term: 1 leader_uuid: "95a95386def14a4fb180faf7d5132622" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } health_report { overall_health: UNKNOWN } } }
Sep 29 16:04:19 dist-test-slave-x07d krb5kdc[22678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697856, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:19.121397 22818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.104876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.20.121.190:32843)
0929 16:04:19.105089 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.105097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.105822 (+ 725us) server_negotiation.cc:407] Connection header received
0929 16:04:19.105973 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.105977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.106132 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.106233 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:19.107608 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.108450 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.109889 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.110549 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.110663 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.114890 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:19.114895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:19.114899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:19.114934 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:19.117969 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:19.119040 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:19.119046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:19.119049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:19.119209 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:19.119675 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:19.119684 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:19.119687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:19.119972 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:19.120202 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.120651 (+ 449us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.121060 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":189,"server-negotiator.queue_time_us":80}
I20260929 16:04:19.121444 23245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.104887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:32843 (local address 127.0.0.1:41058)
0929 16:04:19.105691 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:19.105710 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:19.105862 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:19.106290 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:19.106294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:19.107065 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:19.107373 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.107397 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.108646 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.108651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.109713 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.109726 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.110759 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.110764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.110935 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.111797 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:19.111818 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:19.114638 (+ 2820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:19.118359 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:19.118378 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:19.118387 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:19.118880 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:19.119338 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:19.119342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:19.119347 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:19.119500 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:19.120202 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:19.120218 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:19.120385 (+ 167us) client_negotiation.cc:769] Sending connection context
0929 16:04:19.120571 (+ 186us) client_negotiation.cc:240] Negotiation successful
0929 16:04:19.120924 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":4,"thread_start_us":424,"threads_started":1}
I20260929 16:04:19.127727 22721 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:41058:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 16:04:19.129153 22721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:19.136953 22860 tablet_service.cc:1467] Processing CreateTablet for tablet c2ae53de19ef401c8be71fe21412b04b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c45ad0953f24a60a68e561e9c041b1b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 16:04:19.137322 22999 tablet_service.cc:1467] Processing CreateTablet for tablet c2ae53de19ef401c8be71fe21412b04b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c45ad0953f24a60a68e561e9c041b1b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 16:04:19.138000 22860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ae53de19ef401c8be71fe21412b04b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:19.138200 22999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ae53de19ef401c8be71fe21412b04b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:19.144011 23143 tablet_service.cc:1467] Processing CreateTablet for tablet c2ae53de19ef401c8be71fe21412b04b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1c45ad0953f24a60a68e561e9c041b1b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 16:04:19.145107 23143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2ae53de19ef401c8be71fe21412b04b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:19.145257 23227 tablet_bootstrap.cc:492] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: Bootstrap starting.
I20260929 16:04:19.145861 23228 tablet_bootstrap.cc:492] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: Bootstrap starting.
I20260929 16:04:19.147123 23227 tablet_bootstrap.cc:654] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:19.147603 23228 tablet_bootstrap.cc:654] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:19.149675 23227 tablet_bootstrap.cc:492] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: No bootstrap required, opened a new log
I20260929 16:04:19.149847 23227 ts_tablet_manager.cc:1397] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 16:04:19.150241 23228 tablet_bootstrap.cc:492] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: No bootstrap required, opened a new log
I20260929 16:04:19.150394 23228 ts_tablet_manager.cc:1397] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 16:04:19.150760 23227 raft_consensus.cc:348] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.150996 23227 raft_consensus.cc:374] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:19.151103 23227 raft_consensus.cc:729] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a95386def14a4fb180faf7d5132622, State: Initialized, Role: FOLLOWER
I20260929 16:04:19.151242 23228 raft_consensus.cc:348] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.151470 23228 raft_consensus.cc:374] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:19.151342 23227 consensus_queue.cc:260] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.151563 23228 raft_consensus.cc:729] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8215f6e43b48ebb8e66a779f71f3fd, State: Initialized, Role: FOLLOWER
I20260929 16:04:19.151937 23227 ts_tablet_manager.cc:1428] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260929 16:04:19.151856 23228 consensus_queue.cc:260] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.152329 23234 tablet_replica.cc:439] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } } }
I20260929 16:04:19.152717 23234 tablet_replica.cc:442] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:19.152751 23230 tablet_bootstrap.cc:492] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: Bootstrap starting.
I20260929 16:04:19.153168 23228 ts_tablet_manager.cc:1428] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 16:04:19.153280 23233 tablet_replica.cc:439] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } } }
I20260929 16:04:19.153559 23233 tablet_replica.cc:442] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:19.154477 23230 tablet_bootstrap.cc:654] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:19.156914 23230 tablet_bootstrap.cc:492] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: No bootstrap required, opened a new log
I20260929 16:04:19.157109 23230 ts_tablet_manager.cc:1397] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 16:04:19.157914 23230 raft_consensus.cc:348] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.158233 23230 raft_consensus.cc:374] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:19.158411 23230 raft_consensus.cc:729] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d537755f82cf42f4898789f370d67c3a, State: Initialized, Role: FOLLOWER
I20260929 16:04:19.158694 23230 consensus_queue.cc:260] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.159350 23230 ts_tablet_manager.cc:1428] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 16:04:19.159263 23235 tablet_replica.cc:439] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } } }
I20260929 16:04:19.159677 23235 tablet_replica.cc:442] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:19.312217 23233 raft_consensus.cc:482] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:19.312539 23233 raft_consensus.cc:504] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.313263 23233 leader_election.cc:288] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363), d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383)
I20260929 16:04:19.313946 22880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ae53de19ef401c8be71fe21412b04b" candidate_uuid: "95a95386def14a4fb180faf7d5132622" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" is_pre_election: true
I20260929 16:04:19.314178 23163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ae53de19ef401c8be71fe21412b04b" candidate_uuid: "95a95386def14a4fb180faf7d5132622" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d537755f82cf42f4898789f370d67c3a" is_pre_election: true
I20260929 16:04:19.314347 22880 raft_consensus.cc:2454] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95a95386def14a4fb180faf7d5132622 in term 0.
I20260929 16:04:19.314489 23163 raft_consensus.cc:2454] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95a95386def14a4fb180faf7d5132622 in term 0.
I20260929 16:04:19.314824 22950 leader_election.cc:302] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8215f6e43b48ebb8e66a779f71f3fd, 95a95386def14a4fb180faf7d5132622; no voters:
I20260929 16:04:19.315138 23233 raft_consensus.cc:2781] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:19.315265 23233 raft_consensus.cc:482] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:19.315393 23233 raft_consensus.cc:3037] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:19.317633 23233 raft_consensus.cc:504] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.318325 23233 leader_election.cc:288] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [CANDIDATE]: Term 1 election: Requested vote from peers 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363), d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383)
I20260929 16:04:19.318892 22880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ae53de19ef401c8be71fe21412b04b" candidate_uuid: "95a95386def14a4fb180faf7d5132622" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd"
I20260929 16:04:19.319121 22880 raft_consensus.cc:3037] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:19.319093 23163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2ae53de19ef401c8be71fe21412b04b" candidate_uuid: "95a95386def14a4fb180faf7d5132622" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d537755f82cf42f4898789f370d67c3a"
I20260929 16:04:19.319348 23163 raft_consensus.cc:3037] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:19.321251 22880 raft_consensus.cc:2454] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95a95386def14a4fb180faf7d5132622 in term 1.
I20260929 16:04:19.321544 23163 raft_consensus.cc:2454] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95a95386def14a4fb180faf7d5132622 in term 1.
I20260929 16:04:19.321748 22950 leader_election.cc:302] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8215f6e43b48ebb8e66a779f71f3fd, 95a95386def14a4fb180faf7d5132622; no voters:
I20260929 16:04:19.322058 23233 raft_consensus.cc:2781] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:19.322232 23233 raft_consensus.cc:686] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [term 1 LEADER]: Becoming Leader. State: Replica: 95a95386def14a4fb180faf7d5132622, State: Running, Role: LEADER
I20260929 16:04:19.322525 23233 consensus_queue.cc:237] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.324018 23238 tablet_replica.cc:439] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: TxnStatusTablet state changed. Reason: New leader 95a95386def14a4fb180faf7d5132622. Latest consensus state: current_term: 1 leader_uuid: "95a95386def14a4fb180faf7d5132622" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } } }
I20260929 16:04:19.324316 23238 tablet_replica.cc:442] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: This TxnStatusTablet replica's current role is: LEADER
I20260929 16:04:19.325260 23250 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 16:04:19.325546 23250 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 16:04:19.325801 22723 catalog_manager.cc:5495] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95a95386def14a4fb180faf7d5132622 (127.20.121.130). New cstate: current_term: 1 leader_uuid: "95a95386def14a4fb180faf7d5132622" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } health_report { overall_health: UNKNOWN } } }
I20260929 16:04:19.347169 23245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.340061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39419 (local address 127.0.0.1:49328)
0929 16:04:19.340451 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:19.340472 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:19.340639 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:19.341224 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:19.341229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:19.341242 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:04:19.341432 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.341443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.342679 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.342686 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.344311 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.344327 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.345074 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.345082 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.345310 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.345392 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:04:19.346568 (+ 1176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:04:19.346593 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 16:04:19.346685 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 16:04:19.346845 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216}
I20260929 16:04:19.347219 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.340232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49328 (local address 127.20.121.130:39419)
0929 16:04:19.340858 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.340862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.340881 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:04:19.340954 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.340958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.341032 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.341167 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:19.341618 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.342528 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.344466 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.344907 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.345004 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.345643 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:19.346447 (+ 804us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:19.346631 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.346720 (+ 89us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.346813 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":379,"threads_started":1}
I20260929 16:04:19.355398 22880 raft_consensus.cc:1261] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Refusing update from remote peer 95a95386def14a4fb180faf7d5132622: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:19.355448 23163 raft_consensus.cc:1261] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Refusing update from remote peer 95a95386def14a4fb180faf7d5132622: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:19.356501 23233 consensus_queue.cc:1035] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:19.356909 23238 consensus_queue.cc:1035] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:19.368430 23234 tablet_replica.cc:439] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: TxnStatusTablet state changed. Reason: New leader 95a95386def14a4fb180faf7d5132622. Latest consensus state: current_term: 1 leader_uuid: "95a95386def14a4fb180faf7d5132622" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } } }
I20260929 16:04:19.368877 23234 tablet_replica.cc:442] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:19.371372 23235 tablet_replica.cc:439] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: TxnStatusTablet state changed. Reason: New leader 95a95386def14a4fb180faf7d5132622. Latest consensus state: current_term: 1 leader_uuid: "95a95386def14a4fb180faf7d5132622" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } } }
I20260929 16:04:19.371667 23235 tablet_replica.cc:442] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:19.373193 23254 mvcc.cc:205] Tried to move back new op lower bound from 7334698431909253120 to 7334698431787139072. Current Snapshot: MvccSnapshot[applied={T|T < 7334698431909253120}]
I20260929 16:04:19.375392 23238 tablet_replica.cc:439] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95a95386def14a4fb180faf7d5132622" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } } }
I20260929 16:04:19.375774 23238 tablet_replica.cc:442] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: This TxnStatusTablet replica's current role is: LEADER
I20260929 16:04:19.376051 23238 tablet_replica.cc:439] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95a95386def14a4fb180faf7d5132622" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } } }
I20260929 16:04:19.376323 23238 tablet_replica.cc:442] T c2ae53de19ef401c8be71fe21412b04b P 95a95386def14a4fb180faf7d5132622: This TxnStatusTablet replica's current role is: LEADER
I20260929 16:04:19.379961 23235 tablet_replica.cc:439] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95a95386def14a4fb180faf7d5132622" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } } }
I20260929 16:04:19.380228 23235 tablet_replica.cc:442] T c2ae53de19ef401c8be71fe21412b04b P d537755f82cf42f4898789f370d67c3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:19.384966 23234 tablet_replica.cc:439] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95a95386def14a4fb180faf7d5132622" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } } }
I20260929 16:04:19.385231 23234 tablet_replica.cc:442] T c2ae53de19ef401c8be71fe21412b04b P 3e8215f6e43b48ebb8e66a779f71f3fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:04:19.394272 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.388280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49334 (local address 127.20.121.130:39419)
0929 16:04:19.388439 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.388444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.388946 (+ 502us) server_negotiation.cc:407] Connection header received
0929 16:04:19.389062 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.389068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.389151 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.389254 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:19.389630 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.390865 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.392356 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.393004 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.393131 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.393510 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:19.393640 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:19.393729 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.393851 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.393936 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 16:04:19.413558 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.406224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:51207 (local address 127.20.121.130:39419)
0929 16:04:19.406466 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.406473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.407099 (+ 626us) server_negotiation.cc:407] Connection header received
0929 16:04:19.407266 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.407272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.407356 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.407467 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:19.408028 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.409208 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.411211 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.411945 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.412076 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.412655 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:19.412786 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:19.412867 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.413042 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.413199 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":62}
I20260929 16:04:19.413614 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.406265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39419 (local address 127.20.121.130:51207)
0929 16:04:19.406996 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:19.407012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:19.407145 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:19.407480 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:19.407483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:19.407493 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:04:19.407818 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.407826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.409337 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.409341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.411043 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.411060 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.412225 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.412230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.412367 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.412435 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:04:19.412904 (+ 469us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:04:19.412915 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 16:04:19.413029 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 16:04:19.413226 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":27,"thread_start_us":420,"threads_started":1}
I20260929 16:04:19.424410 22880 raft_consensus.cc:1261] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Refusing update from remote peer 95a95386def14a4fb180faf7d5132622: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:19.424481 23162 raft_consensus.cc:1261] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Refusing update from remote peer 95a95386def14a4fb180faf7d5132622: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:19.425040 23238 consensus_queue.cc:1035] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:19.425393 23233 consensus_queue.cc:1035] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:19.435125 23255 mvcc.cc:205] Tried to move back new op lower bound from 7334698432197152768 to 7334698430602780672. Current Snapshot: MvccSnapshot[applied={T|T < 7334698432197152768}]
I20260929 16:04:19.495995 22721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:19.496452 22721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:19.500319 22721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be: Sending DeleteTablet for 3 replicas of tablet d19ff3e5fc514243914374d33106a7e7
I20260929 16:04:19.501636 22860 tablet_service.cc:1514] Processing DeleteTablet for tablet d19ff3e5fc514243914374d33106a7e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:19 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:51188
I20260929 16:04:19.502190 22998 tablet_service.cc:1514] Processing DeleteTablet for tablet d19ff3e5fc514243914374d33106a7e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:19 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:49320
I20260929 16:04:19.502743 23143 tablet_service.cc:1514] Processing DeleteTablet for tablet d19ff3e5fc514243914374d33106a7e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:19 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:36110
I20260929 16:04:19.503772 23273 tablet_replica.cc:330] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: stopping tablet replica
I20260929 16:04:19.503513 22721 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:04:19.504482 22721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:19.504527 23273 raft_consensus.cc:2229] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:19.505097 23273 raft_consensus.cc:2258] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:19.507200 23273 ts_tablet_manager.cc:1905] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:19.515462 23273 ts_tablet_manager.cc:1918] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:04:19.515762 23273 log.cc:1196] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/d19ff3e5fc514243914374d33106a7e7
I20260929 16:04:19.516381 23273 ts_tablet_manager.cc:1939] T d19ff3e5fc514243914374d33106a7e7 P d537755f82cf42f4898789f370d67c3a: Deleting consensus metadata
I20260929 16:04:19.517805 23271 tablet_replica.cc:330] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: stopping tablet replica
I20260929 16:04:19.518440 23271 raft_consensus.cc:2229] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:19.519027 23271 raft_consensus.cc:2258] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:19.519392 22707 catalog_manager.cc:4841] TS d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383): tablet d19ff3e5fc514243914374d33106a7e7 (table test-table [id=8e0d6c15e2be472194d772ab1ccf0f01]) successfully deleted
I20260929 16:04:19.519393 22999 tablet_service.cc:1467] Processing CreateTablet for tablet f2be12593b344884aa04e88fabee8014 (DEFAULT_TABLE table=test-table [id=d56cb27fe5b542b7b662cdaddd2cb98b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:19.519863 22999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2be12593b344884aa04e88fabee8014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:19.520529 23143 tablet_service.cc:1467] Processing CreateTablet for tablet f2be12593b344884aa04e88fabee8014 (DEFAULT_TABLE table=test-table [id=d56cb27fe5b542b7b662cdaddd2cb98b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:19.520956 23143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2be12593b344884aa04e88fabee8014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:19.521273 23271 ts_tablet_manager.cc:1905] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:19.526129 22860 tablet_service.cc:1467] Processing CreateTablet for tablet f2be12593b344884aa04e88fabee8014 (DEFAULT_TABLE table=test-table [id=d56cb27fe5b542b7b662cdaddd2cb98b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:19.526619 22860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2be12593b344884aa04e88fabee8014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:19.526845 23227 tablet_bootstrap.cc:492] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622: Bootstrap starting.
I20260929 16:04:19.528406 23227 tablet_bootstrap.cc:654] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:19.531190 23271 ts_tablet_manager.cc:1918] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:04:19.531394 23271 log.cc:1196] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/d19ff3e5fc514243914374d33106a7e7
I20260929 16:04:19.531983 23271 ts_tablet_manager.cc:1939] T d19ff3e5fc514243914374d33106a7e7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting consensus metadata
I20260929 16:04:19.535022 22707 catalog_manager.cc:4841] TS 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363): tablet d19ff3e5fc514243914374d33106a7e7 (table test-table [id=8e0d6c15e2be472194d772ab1ccf0f01]) successfully deleted
I20260929 16:04:19.536816 23230 tablet_bootstrap.cc:492] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a: Bootstrap starting.
I20260929 16:04:19.538733 23230 tablet_bootstrap.cc:654] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:19.540306 23228 tablet_bootstrap.cc:492] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd: Bootstrap starting.
I20260929 16:04:19.540457 23227 tablet_bootstrap.cc:492] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622: No bootstrap required, opened a new log
I20260929 16:04:19.540642 23227 ts_tablet_manager.cc:1397] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260929 16:04:19.541431 23227 raft_consensus.cc:348] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.541697 23227 raft_consensus.cc:374] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:19.541779 23227 raft_consensus.cc:729] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a95386def14a4fb180faf7d5132622, State: Initialized, Role: FOLLOWER
I20260929 16:04:19.542089 23227 consensus_queue.cc:260] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.542544 23228 tablet_bootstrap.cc:654] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:19.546336 23227 ts_tablet_manager.cc:1428] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260929 16:04:19.549077 23228 tablet_bootstrap.cc:492] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd: No bootstrap required, opened a new log
I20260929 16:04:19.549322 23228 ts_tablet_manager.cc:1397] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 16:04:19.550067 23272 tablet_replica.cc:330] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: stopping tablet replica
I20260929 16:04:19.550338 23228 raft_consensus.cc:348] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.550545 23272 raft_consensus.cc:2229] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:04:19.550697 23228 raft_consensus.cc:374] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:19.550806 23228 raft_consensus.cc:729] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8215f6e43b48ebb8e66a779f71f3fd, State: Initialized, Role: FOLLOWER
I20260929 16:04:19.551167 23272 raft_consensus.cc:2258] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:19.551198 23228 consensus_queue.cc:260] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.553117 23272 ts_tablet_manager.cc:1905] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:19.553335 23230 tablet_bootstrap.cc:492] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a: No bootstrap required, opened a new log
I20260929 16:04:19.553531 23230 ts_tablet_manager.cc:1397] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.007s
I20260929 16:04:19.554378 23228 ts_tablet_manager.cc:1428] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260929 16:04:19.554407 23230 raft_consensus.cc:348] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.554668 23230 raft_consensus.cc:374] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:19.554764 23230 raft_consensus.cc:729] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d537755f82cf42f4898789f370d67c3a, State: Initialized, Role: FOLLOWER
I20260929 16:04:19.555197 23234 raft_consensus.cc:482] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:19.555054 23230 consensus_queue.cc:260] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.555374 23234 raft_consensus.cc:504] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.556232 23230 ts_tablet_manager.cc:1428] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 16:04:19.557217 23234 leader_election.cc:288] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419), d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383)
I20260929 16:04:19.565533 23272 ts_tablet_manager.cc:1918] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:04:19.565723 23272 log.cc:1196] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/d19ff3e5fc514243914374d33106a7e7
I20260929 16:04:19.566495 23272 ts_tablet_manager.cc:1939] T d19ff3e5fc514243914374d33106a7e7 P 95a95386def14a4fb180faf7d5132622: Deleting consensus metadata
I20260929 16:04:19.568967 22708 catalog_manager.cc:4841] TS 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419): tablet d19ff3e5fc514243914374d33106a7e7 (table test-table [id=8e0d6c15e2be472194d772ab1ccf0f01]) successfully deleted
I20260929 16:04:19.577657 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.558728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:56183 (local address 127.20.121.130:39419)
0929 16:04:19.559029 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.559039 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.570771 (+ 11732us) server_negotiation.cc:407] Connection header received
0929 16:04:19.570989 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.570994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.571052 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.571165 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:19.571945 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.573060 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.575622 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.576593 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.576781 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.576860 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.577182 (+ 322us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.577339 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260929 16:04:19.577883 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.557688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:34383 (local address 127.20.121.129:50873)
0929 16:04:19.570055 (+ 12367us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:19.570084 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:19.570459 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:19.570973 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:19.570978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:19.571002 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:19.571321 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.571335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.572697 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.572705 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.575166 (+ 2461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.575181 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.577000 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.577007 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.577190 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.577222 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 16:04:19.577309 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 16:04:19.577465 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12199,"mutex_wait_us":9,"thread_start_us":12534,"threads_started":1}
I20260929 16:04:19.578447 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.557759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39419 (local address 127.20.121.129:56183)
0929 16:04:19.570501 (+ 12742us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:19.570664 (+ 163us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:19.570822 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:19.571184 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:19.571188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:19.571212 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:19.571742 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.571757 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.573210 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.573216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.575463 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.575473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.576771 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.576777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.576961 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.576997 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 16:04:19.577857 (+ 860us) client_negotiation.cc:240] Negotiation successful
0929 16:04:19.578031 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12618,"mutex_wait_us":55,"thread_start_us":11988,"threads_started":1}
I20260929 16:04:19.578717 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.558351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:50873 (local address 127.20.121.131:34383)
0929 16:04:19.569569 (+ 11218us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.569576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.570316 (+ 740us) server_negotiation.cc:407] Connection header received
0929 16:04:19.570637 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.570643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.570719 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.570801 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:19.571537 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.572542 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.575343 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.576622 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.576770 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.576845 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.578199 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.578344 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11080,"thread_start_us":10940,"threads_started":1}
I20260929 16:04:19.579418 23163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2be12593b344884aa04e88fabee8014" candidate_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d537755f82cf42f4898789f370d67c3a" is_pre_election: true
I20260929 16:04:19.579455 23019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2be12593b344884aa04e88fabee8014" candidate_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a95386def14a4fb180faf7d5132622" is_pre_election: true
I20260929 16:04:19.579751 23163 raft_consensus.cc:2454] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e8215f6e43b48ebb8e66a779f71f3fd in term 0.
I20260929 16:04:19.579877 23019 raft_consensus.cc:2454] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e8215f6e43b48ebb8e66a779f71f3fd in term 0.
I20260929 16:04:19.580327 22810 leader_election.cc:302] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8215f6e43b48ebb8e66a779f71f3fd, d537755f82cf42f4898789f370d67c3a; no voters:
I20260929 16:04:19.580792 23234 raft_consensus.cc:2781] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:19.580925 23234 raft_consensus.cc:482] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:19.581020 23234 raft_consensus.cc:3037] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:19.584199 23234 raft_consensus.cc:504] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.584977 23234 leader_election.cc:288] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [CANDIDATE]: Term 1 election: Requested vote from peers 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419), d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383)
I20260929 16:04:19.585525 23163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2be12593b344884aa04e88fabee8014" candidate_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d537755f82cf42f4898789f370d67c3a"
I20260929 16:04:19.585793 23163 raft_consensus.cc:3037] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:19.586318 23019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2be12593b344884aa04e88fabee8014" candidate_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a95386def14a4fb180faf7d5132622"
I20260929 16:04:19.586556 23019 raft_consensus.cc:3037] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:19.588842 23163 raft_consensus.cc:2454] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8215f6e43b48ebb8e66a779f71f3fd in term 1.
I20260929 16:04:19.589092 23019 raft_consensus.cc:2454] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8215f6e43b48ebb8e66a779f71f3fd in term 1.
I20260929 16:04:19.589385 22810 leader_election.cc:302] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8215f6e43b48ebb8e66a779f71f3fd, d537755f82cf42f4898789f370d67c3a; no voters:
I20260929 16:04:19.589882 23234 raft_consensus.cc:2781] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:19.590555 23234 raft_consensus.cc:686] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 LEADER]: Becoming Leader. State: Replica: 3e8215f6e43b48ebb8e66a779f71f3fd, State: Running, Role: LEADER
I20260929 16:04:19.591265 23234 consensus_queue.cc:237] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:19.596362 22722 catalog_manager.cc:5495] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129). New cstate: current_term: 1 leader_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } health_report { overall_health: UNKNOWN } } }
I20260929 16:04:19.628726 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.622165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51192 (local address 127.20.121.129:40363)
0929 16:04:19.622810 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.622816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.622838 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:04:19.622938 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.622944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.623035 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.623265 (+ 230us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:19.623768 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.624662 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.626470 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.626863 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.626962 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.627460 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:19.628017 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:19.628135 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.628335 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.628402 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":362,"threads_started":1}
I20260929 16:04:19.646937 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.640192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39419 (local address 127.20.121.129:35013)
0929 16:04:19.640899 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:19.640915 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:19.641104 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:19.641477 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:19.641480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:19.641489 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:04:19.641749 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.641756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.643268 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.643272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.644388 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.644401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.645541 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.645585 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.645760 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.645841 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:04:19.646417 (+ 576us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:04:19.646433 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 16:04:19.646519 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 16:04:19.646637 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":412,"threads_started":1}
I20260929 16:04:19.646968 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.640190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:35013 (local address 127.20.121.130:39419)
0929 16:04:19.640433 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.640440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.641019 (+ 579us) server_negotiation.cc:407] Connection header received
0929 16:04:19.641249 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.641255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.641348 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.641468 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:19.641908 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.643093 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.644556 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.645352 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.645516 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.646078 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:19.646261 (+ 183us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:19.646351 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.646565 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.646662 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 16:04:19.655876 23163 raft_consensus.cc:1261] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Refusing update from remote peer 3e8215f6e43b48ebb8e66a779f71f3fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:19.656035 23019 raft_consensus.cc:1261] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Refusing update from remote peer 3e8215f6e43b48ebb8e66a779f71f3fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:19.656585 23280 consensus_queue.cc:1035] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:19.656960 23234 consensus_queue.cc:1035] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:19.686233 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.679782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:37323 (local address 127.20.121.129:40363)
0929 16:04:19.679916 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:19.679922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:19.680007 (+ 85us) server_negotiation.cc:407] Connection header received
0929 16:04:19.680177 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:19.680182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:19.680261 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:19.680352 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:19.680899 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.681951 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.684125 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.684784 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.684927 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.685406 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:19.685547 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:19.685637 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:19.685857 (+ 220us) server_negotiation.cc:299] Negotiation successful
0929 16:04:19.685941 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 16:04:19.686407 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:19.679725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40363 (local address 127.20.121.130:37323)
0929 16:04:19.679899 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:19.679918 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:19.680045 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:19.680539 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:19.680543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:19.680555 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:04:19.680743 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.680753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.682147 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.682155 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.683956 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:19.683971 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:19.684967 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:19.684973 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:19.685169 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:19.685221 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:04:19.685722 (+ 501us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:04:19.685734 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 16:04:19.685816 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 16:04:19.686008 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 16:04:19.687625 23255 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 16:04:21.297470 22722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:21.297715 22722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:21.302938 22722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be: Sending DeleteTablet for 3 replicas of tablet f2be12593b344884aa04e88fabee8014
I20260929 16:04:21.304020 22860 tablet_service.cc:1514] Processing DeleteTablet for tablet f2be12593b344884aa04e88fabee8014 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:21 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:51188
I20260929 16:04:21.304090 22998 tablet_service.cc:1514] Processing DeleteTablet for tablet f2be12593b344884aa04e88fabee8014 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:21 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:49320
I20260929 16:04:21.304195 23143 tablet_service.cc:1514] Processing DeleteTablet for tablet f2be12593b344884aa04e88fabee8014 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:21 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:36110
I20260929 16:04:21.315234 23313 tablet_replica.cc:330] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd: stopping tablet replica
I20260929 16:04:21.315587 23313 raft_consensus.cc:2229] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:04:21.316438 23313 raft_consensus.cc:2258] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:21.316896 23314 tablet_replica.cc:330] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a: stopping tablet replica
I20260929 16:04:21.317209 23312 tablet_replica.cc:330] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622: stopping tablet replica
I20260929 16:04:21.317265 23314 raft_consensus.cc:2229] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:21.317444 23312 raft_consensus.cc:2229] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:21.317582 23314 raft_consensus.cc:2258] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:21.317761 23312 raft_consensus.cc:2258] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:21.318687 23313 ts_tablet_manager.cc:1905] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:21.318764 23312 ts_tablet_manager.cc:1905] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:21.320430 23314 ts_tablet_manager.cc:1905] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:21.325268 23312 ts_tablet_manager.cc:1918] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 16:04:21.325429 23312 log.cc:1196] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/f2be12593b344884aa04e88fabee8014
I20260929 16:04:21.325889 23312 ts_tablet_manager.cc:1939] T f2be12593b344884aa04e88fabee8014 P 95a95386def14a4fb180faf7d5132622: Deleting consensus metadata
I20260929 16:04:21.327239 22708 catalog_manager.cc:4841] TS 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419): tablet f2be12593b344884aa04e88fabee8014 (table test-table [id=d56cb27fe5b542b7b662cdaddd2cb98b]) successfully deleted
I20260929 16:04:21.327574 23313 ts_tablet_manager.cc:1918] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 16:04:21.327764 23313 log.cc:1196] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/f2be12593b344884aa04e88fabee8014
I20260929 16:04:21.327737 23314 ts_tablet_manager.cc:1918] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 16:04:21.327893 23314 log.cc:1196] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/f2be12593b344884aa04e88fabee8014
I20260929 16:04:21.328315 23313 ts_tablet_manager.cc:1939] T f2be12593b344884aa04e88fabee8014 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting consensus metadata
I20260929 16:04:21.328336 23314 ts_tablet_manager.cc:1939] T f2be12593b344884aa04e88fabee8014 P d537755f82cf42f4898789f370d67c3a: Deleting consensus metadata
I20260929 16:04:21.329747 22707 catalog_manager.cc:4841] TS 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363): tablet f2be12593b344884aa04e88fabee8014 (table test-table [id=d56cb27fe5b542b7b662cdaddd2cb98b]) successfully deleted
I20260929 16:04:21.330093 22707 catalog_manager.cc:4841] TS d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383): tablet f2be12593b344884aa04e88fabee8014 (table test-table [id=d56cb27fe5b542b7b662cdaddd2cb98b]) successfully deleted
Sep 29 16:04:21 dist-test-slave-x07d krb5kdc[22678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 16:04:21 dist-test-slave-x07d krb5kdc[22678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:21.375123 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.355977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.20.121.190:32843)
0929 16:04:21.356810 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:21.356816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:21.356857 (+ 41us) server_negotiation.cc:407] Connection header received
0929 16:04:21.356970 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:21.356978 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:21.357078 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:21.357232 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:21.359323 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.360219 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.361803 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.362338 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.362456 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.368718 (+ 6262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:21.368725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:21.368729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:21.368775 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:21.372070 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:21.373136 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:21.373140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:21.373143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:21.373280 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:21.373661 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:21.373665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:21.373667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:21.373938 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:21.374269 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:21.374562 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 16:04:21.374745 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":435,"threads_started":1}
I20260929 16:04:21.379798 22722 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:04:21.380493 22722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:21.388945 22860 tablet_service.cc:1467] Processing CreateTablet for tablet 8bbef0701362409795fb2926da86dfd7 (DEFAULT_TABLE table=test-table [id=8c40e77118384082bfc37a737a7da271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:21.389051 22998 tablet_service.cc:1467] Processing CreateTablet for tablet 8bbef0701362409795fb2926da86dfd7 (DEFAULT_TABLE table=test-table [id=8c40e77118384082bfc37a737a7da271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:21.389201 23143 tablet_service.cc:1467] Processing CreateTablet for tablet 8bbef0701362409795fb2926da86dfd7 (DEFAULT_TABLE table=test-table [id=8c40e77118384082bfc37a737a7da271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:21.389555 22998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bbef0701362409795fb2926da86dfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:21.389564 22860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bbef0701362409795fb2926da86dfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:21.389747 23143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bbef0701362409795fb2926da86dfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:21.398082 23325 tablet_bootstrap.cc:492] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Bootstrap starting.
I20260929 16:04:21.398326 23324 tablet_bootstrap.cc:492] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622: Bootstrap starting.
I20260929 16:04:21.400305 23325 tablet_bootstrap.cc:654] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:21.401252 23323 tablet_bootstrap.cc:492] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a: Bootstrap starting.
I20260929 16:04:21.401862 23324 tablet_bootstrap.cc:654] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:21.402777 23325 tablet_bootstrap.cc:492] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd: No bootstrap required, opened a new log
I20260929 16:04:21.402990 23325 ts_tablet_manager.cc:1397] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260929 16:04:21.404289 23323 tablet_bootstrap.cc:654] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:21.405103 23325 raft_consensus.cc:348] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.405503 23325 raft_consensus.cc:374] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:21.405612 23325 raft_consensus.cc:729] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8215f6e43b48ebb8e66a779f71f3fd, State: Initialized, Role: FOLLOWER
I20260929 16:04:21.405907 23325 consensus_queue.cc:260] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.406083 23324 tablet_bootstrap.cc:492] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622: No bootstrap required, opened a new log
I20260929 16:04:21.406276 23324 ts_tablet_manager.cc:1397] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 16:04:21.406606 23325 ts_tablet_manager.cc:1428] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:21.407313 23324 raft_consensus.cc:348] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.409392 23324 raft_consensus.cc:374] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:21.409544 23324 raft_consensus.cc:729] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a95386def14a4fb180faf7d5132622, State: Initialized, Role: FOLLOWER
I20260929 16:04:21.409933 23324 consensus_queue.cc:260] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.410811 23324 ts_tablet_manager.cc:1428] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260929 16:04:21.412606 23323 tablet_bootstrap.cc:492] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a: No bootstrap required, opened a new log
I20260929 16:04:21.412791 23323 ts_tablet_manager.cc:1397] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260929 16:04:21.413774 23323 raft_consensus.cc:348] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.414088 23323 raft_consensus.cc:374] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:21.414198 23323 raft_consensus.cc:729] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d537755f82cf42f4898789f370d67c3a, State: Initialized, Role: FOLLOWER
I20260929 16:04:21.414456 23323 consensus_queue.cc:260] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.415403 23323 ts_tablet_manager.cc:1428] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 16:04:21.468797 23302 raft_consensus.cc:482] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:21.469019 23302 raft_consensus.cc:504] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.469902 23302 leader_election.cc:288] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419), d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383)
I20260929 16:04:21.470702 23019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbef0701362409795fb2926da86dfd7" candidate_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a95386def14a4fb180faf7d5132622" is_pre_election: true
I20260929 16:04:21.470973 23019 raft_consensus.cc:2454] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e8215f6e43b48ebb8e66a779f71f3fd in term 0.
I20260929 16:04:21.470820 23162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbef0701362409795fb2926da86dfd7" candidate_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d537755f82cf42f4898789f370d67c3a" is_pre_election: true
I20260929 16:04:21.471282 23162 raft_consensus.cc:2454] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e8215f6e43b48ebb8e66a779f71f3fd in term 0.
I20260929 16:04:21.471570 22811 leader_election.cc:302] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8215f6e43b48ebb8e66a779f71f3fd, 95a95386def14a4fb180faf7d5132622; no voters:
I20260929 16:04:21.471864 23302 raft_consensus.cc:2781] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:21.471980 23302 raft_consensus.cc:482] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:21.472082 23302 raft_consensus.cc:3037] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:21.474737 23302 raft_consensus.cc:504] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.475414 23302 leader_election.cc:288] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [CANDIDATE]: Term 1 election: Requested vote from peers 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419), d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383)
I20260929 16:04:21.476310 23019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbef0701362409795fb2926da86dfd7" candidate_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a95386def14a4fb180faf7d5132622"
I20260929 16:04:21.476306 23162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bbef0701362409795fb2926da86dfd7" candidate_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d537755f82cf42f4898789f370d67c3a"
I20260929 16:04:21.476614 23019 raft_consensus.cc:3037] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:21.476647 23162 raft_consensus.cc:3037] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:21.480260 23162 raft_consensus.cc:2454] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8215f6e43b48ebb8e66a779f71f3fd in term 1.
I20260929 16:04:21.480280 23019 raft_consensus.cc:2454] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e8215f6e43b48ebb8e66a779f71f3fd in term 1.
I20260929 16:04:21.481002 22810 leader_election.cc:302] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8215f6e43b48ebb8e66a779f71f3fd, d537755f82cf42f4898789f370d67c3a; no voters:
I20260929 16:04:21.481442 23302 raft_consensus.cc:2781] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:21.481621 23302 raft_consensus.cc:686] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 LEADER]: Becoming Leader. State: Replica: 3e8215f6e43b48ebb8e66a779f71f3fd, State: Running, Role: LEADER
I20260929 16:04:21.482051 23302 consensus_queue.cc:237] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.485579 22722 catalog_manager.cc:5495] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129). New cstate: current_term: 1 leader_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } health_report { overall_health: UNKNOWN } } }
I20260929 16:04:21.518626 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.513072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51200 (local address 127.20.121.129:40363)
0929 16:04:21.513810 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:21.513814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:21.513833 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:04:21.513893 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:21.513897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:21.513966 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:21.514156 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:21.514621 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.515412 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.516771 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.517219 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.517352 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.517796 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:21.517916 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:21.518043 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:21.518203 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 16:04:21.518283 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":380,"threads_started":1}
I20260929 16:04:21.530926 23162 raft_consensus.cc:1261] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Refusing update from remote peer 3e8215f6e43b48ebb8e66a779f71f3fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:21.530972 23019 raft_consensus.cc:1261] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Refusing update from remote peer 3e8215f6e43b48ebb8e66a779f71f3fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:21.531634 23305 consensus_queue.cc:1035] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:21.531989 23302 consensus_queue.cc:1035] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:21.585049 22721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:21.585323 22721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:21.589006 22721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be: Sending DeleteTablet for 3 replicas of tablet 8bbef0701362409795fb2926da86dfd7
I20260929 16:04:21.590210 22998 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bbef0701362409795fb2926da86dfd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:21 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:49320
I20260929 16:04:21.590176 23143 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bbef0701362409795fb2926da86dfd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:21 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:36110
I20260929 16:04:21.590253 22860 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bbef0701362409795fb2926da86dfd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:21 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:51188
I20260929 16:04:21.590487 23312 tablet_replica.cc:330] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622: stopping tablet replica
I20260929 16:04:21.590498 23314 tablet_replica.cc:330] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a: stopping tablet replica
I20260929 16:04:21.590634 23313 tablet_replica.cc:330] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd: stopping tablet replica
I20260929 16:04:21.590687 23312 raft_consensus.cc:2229] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:21.590735 23314 raft_consensus.cc:2229] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:21.590848 23313 raft_consensus.cc:2229] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:04:21.590898 23312 raft_consensus.cc:2258] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:21.590926 23314 raft_consensus.cc:2258] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:21.591265 23313 raft_consensus.cc:2258] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:21.592118 23312 ts_tablet_manager.cc:1905] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:21.592106 23314 ts_tablet_manager.cc:1905] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:21.592597 23313 ts_tablet_manager.cc:1905] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:21.592669 22721 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:04:21.593564 22721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:21.600404 23314 ts_tablet_manager.cc:1918] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:04:21.600396 23313 ts_tablet_manager.cc:1918] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:04:21.600591 23314 log.cc:1196] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/8bbef0701362409795fb2926da86dfd7
I20260929 16:04:21.600625 23313 log.cc:1196] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/8bbef0701362409795fb2926da86dfd7
I20260929 16:04:21.601071 23314 ts_tablet_manager.cc:1939] T 8bbef0701362409795fb2926da86dfd7 P d537755f82cf42f4898789f370d67c3a: Deleting consensus metadata
I20260929 16:04:21.601133 23313 ts_tablet_manager.cc:1939] T 8bbef0701362409795fb2926da86dfd7 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting consensus metadata
I20260929 16:04:21.602293 22707 catalog_manager.cc:4841] TS d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383): tablet 8bbef0701362409795fb2926da86dfd7 (table test-table [id=8c40e77118384082bfc37a737a7da271]) successfully deleted
I20260929 16:04:21.602579 22707 catalog_manager.cc:4841] TS 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363): tablet 8bbef0701362409795fb2926da86dfd7 (table test-table [id=8c40e77118384082bfc37a737a7da271]) successfully deleted
I20260929 16:04:21.604336 23312 ts_tablet_manager.cc:1918] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:04:21.604583 23312 log.cc:1196] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/8bbef0701362409795fb2926da86dfd7
I20260929 16:04:21.605149 23312 ts_tablet_manager.cc:1939] T 8bbef0701362409795fb2926da86dfd7 P 95a95386def14a4fb180faf7d5132622: Deleting consensus metadata
I20260929 16:04:21.607161 22708 catalog_manager.cc:4841] TS 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419): tablet 8bbef0701362409795fb2926da86dfd7 (table test-table [id=8c40e77118384082bfc37a737a7da271]) successfully deleted
I20260929 16:04:21.608175 22998 tablet_service.cc:1467] Processing CreateTablet for tablet 155ba6733fe34e8ba4f03310937ad5d9 (DEFAULT_TABLE table=test-table [id=42a93af7d64a46caa45bc7ee581f1f9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:21.608245 22860 tablet_service.cc:1467] Processing CreateTablet for tablet 155ba6733fe34e8ba4f03310937ad5d9 (DEFAULT_TABLE table=test-table [id=42a93af7d64a46caa45bc7ee581f1f9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:21.608318 23143 tablet_service.cc:1467] Processing CreateTablet for tablet 155ba6733fe34e8ba4f03310937ad5d9 (DEFAULT_TABLE table=test-table [id=42a93af7d64a46caa45bc7ee581f1f9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:21.608731 22998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 155ba6733fe34e8ba4f03310937ad5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:21.608757 23143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 155ba6733fe34e8ba4f03310937ad5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:21.608783 22860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 155ba6733fe34e8ba4f03310937ad5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:21.615959 23324 tablet_bootstrap.cc:492] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622: Bootstrap starting.
I20260929 16:04:21.616274 23325 tablet_bootstrap.cc:492] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd: Bootstrap starting.
I20260929 16:04:21.616467 23323 tablet_bootstrap.cc:492] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a: Bootstrap starting.
I20260929 16:04:21.617728 23324 tablet_bootstrap.cc:654] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:21.618201 23323 tablet_bootstrap.cc:654] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:21.618287 23325 tablet_bootstrap.cc:654] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:21.625648 23323 tablet_bootstrap.cc:492] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a: No bootstrap required, opened a new log
I20260929 16:04:21.625646 23324 tablet_bootstrap.cc:492] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622: No bootstrap required, opened a new log
I20260929 16:04:21.625841 23324 ts_tablet_manager.cc:1397] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260929 16:04:21.625841 23323 ts_tablet_manager.cc:1397] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260929 16:04:21.626648 23324 raft_consensus.cc:348] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.626648 23323 raft_consensus.cc:348] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.626911 23324 raft_consensus.cc:374] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:21.626917 23323 raft_consensus.cc:374] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:21.627004 23323 raft_consensus.cc:729] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d537755f82cf42f4898789f370d67c3a, State: Initialized, Role: FOLLOWER
I20260929 16:04:21.627004 23324 raft_consensus.cc:729] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95a95386def14a4fb180faf7d5132622, State: Initialized, Role: FOLLOWER
I20260929 16:04:21.627269 23324 consensus_queue.cc:260] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.627324 23323 consensus_queue.cc:260] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.628067 23323 ts_tablet_manager.cc:1428] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 16:04:21.629307 23324 ts_tablet_manager.cc:1428] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260929 16:04:21.630251 23325 tablet_bootstrap.cc:492] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd: No bootstrap required, opened a new log
I20260929 16:04:21.630474 23325 ts_tablet_manager.cc:1397] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.009s
I20260929 16:04:21.631790 23325 raft_consensus.cc:348] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.632086 23325 raft_consensus.cc:374] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:21.632182 23325 raft_consensus.cc:729] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8215f6e43b48ebb8e66a779f71f3fd, State: Initialized, Role: FOLLOWER
I20260929 16:04:21.632467 23325 consensus_queue.cc:260] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.633186 23325 ts_tablet_manager.cc:1428] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 16:04:21.646837 23329 raft_consensus.cc:482] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:21.647344 23329 raft_consensus.cc:504] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.649619 23329 leader_election.cc:288] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363), 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419)
I20260929 16:04:21.658569 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.650181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:39355 (local address 127.20.121.130:39419)
0929 16:04:21.652404 (+ 2223us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:21.652410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:21.652438 (+ 28us) server_negotiation.cc:407] Connection header received
0929 16:04:21.652612 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:21.652618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:21.652695 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:21.652802 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:21.653485 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.654334 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.656427 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.657385 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.657602 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.657680 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:21.657991 (+ 311us) server_negotiation.cc:299] Negotiation successful
0929 16:04:21.658185 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2096,"thread_start_us":1926,"threads_started":1}
I20260929 16:04:21.658784 23339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.649938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40363 (local address 127.20.121.131:54487)
0929 16:04:21.651263 (+ 1325us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:21.651294 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:21.651621 (+ 327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:21.652353 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:21.652361 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:21.652413 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:21.652739 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:21.652749 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.654069 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.654074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:21.656195 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:21.656210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.657687 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.657693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:21.657866 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.657892 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 16:04:21.658004 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 16:04:21.658191 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1105,"mutex_wait_us":54,"spinlock_wait_cycles":5504,"thread_start_us":908,"threads_started":1}
I20260929 16:04:21.658847 23341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.650074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39419 (local address 127.20.121.131:39355)
0929 16:04:21.652191 (+ 2117us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:21.652216 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:21.652408 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:21.652824 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:21.652829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:21.652858 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:21.653299 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:21.653309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.654464 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.654476 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:21.656246 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:21.656261 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.657610 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.657616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:21.657788 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.657849 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 16:04:21.657969 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 16:04:21.658151 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1960,"thread_start_us":1262,"threads_started":1}
I20260929 16:04:21.658624 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.650210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:54487 (local address 127.20.121.129:40363)
0929 16:04:21.650624 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:21.650634 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:21.651439 (+ 805us) server_negotiation.cc:407] Connection header received
0929 16:04:21.651871 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:21.651881 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:21.651994 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:21.652078 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:21.652981 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.653891 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.656361 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.657447 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.657639 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.657745 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:21.658061 (+ 316us) server_negotiation.cc:299] Negotiation successful
0929 16:04:21.658212 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208}
I20260929 16:04:21.659564 23019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "155ba6733fe34e8ba4f03310937ad5d9" candidate_uuid: "d537755f82cf42f4898789f370d67c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a95386def14a4fb180faf7d5132622" is_pre_election: true
I20260929 16:04:21.659870 23019 raft_consensus.cc:2454] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d537755f82cf42f4898789f370d67c3a in term 0.
I20260929 16:04:21.659883 22880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "155ba6733fe34e8ba4f03310937ad5d9" candidate_uuid: "d537755f82cf42f4898789f370d67c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" is_pre_election: true
I20260929 16:04:21.660135 22880 raft_consensus.cc:2454] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d537755f82cf42f4898789f370d67c3a in term 0.
I20260929 16:04:21.660465 23090 leader_election.cc:302] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95a95386def14a4fb180faf7d5132622, d537755f82cf42f4898789f370d67c3a; no voters:
I20260929 16:04:21.660935 23329 raft_consensus.cc:2781] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:21.661104 23329 raft_consensus.cc:482] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:21.661224 23329 raft_consensus.cc:3037] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:21.664103 23329 raft_consensus.cc:504] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.664960 23329 leader_election.cc:288] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [CANDIDATE]: Term 1 election: Requested vote from peers 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363), 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419)
I20260929 16:04:21.665542 22880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "155ba6733fe34e8ba4f03310937ad5d9" candidate_uuid: "d537755f82cf42f4898789f370d67c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd"
I20260929 16:04:21.665657 23019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "155ba6733fe34e8ba4f03310937ad5d9" candidate_uuid: "d537755f82cf42f4898789f370d67c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95a95386def14a4fb180faf7d5132622"
I20260929 16:04:21.665817 22880 raft_consensus.cc:3037] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:21.665948 23019 raft_consensus.cc:3037] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:21.668269 22880 raft_consensus.cc:2454] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d537755f82cf42f4898789f370d67c3a in term 1.
I20260929 16:04:21.668751 23089 leader_election.cc:302] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e8215f6e43b48ebb8e66a779f71f3fd, d537755f82cf42f4898789f370d67c3a; no voters:
I20260929 16:04:21.669122 23329 raft_consensus.cc:2781] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:21.669168 23019 raft_consensus.cc:2454] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d537755f82cf42f4898789f370d67c3a in term 1.
I20260929 16:04:21.669727 23329 raft_consensus.cc:686] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 1 LEADER]: Becoming Leader. State: Replica: d537755f82cf42f4898789f370d67c3a, State: Running, Role: LEADER
I20260929 16:04:21.670264 23329 consensus_queue.cc:237] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } }
I20260929 16:04:21.674247 22721 catalog_manager.cc:5495] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a reported cstate change: term changed from 0 to 1, leader changed from <none> to d537755f82cf42f4898789f370d67c3a (127.20.121.131). New cstate: current_term: 1 leader_uuid: "d537755f82cf42f4898789f370d67c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d537755f82cf42f4898789f370d67c3a" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 34383 } health_report { overall_health: HEALTHY } } }
I20260929 16:04:21.701397 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.692065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36122 (local address 127.20.121.131:34383)
0929 16:04:21.694600 (+ 2535us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:21.694606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:21.694629 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:04:21.694771 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:21.694775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:21.694864 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:21.695091 (+ 227us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:21.695616 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.697037 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.698609 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.699067 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.699217 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.699781 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:21.700612 (+ 831us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:21.700812 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:21.700866 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 16:04:21.700967 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2384,"thread_start_us":2206,"threads_started":1}
I20260929 16:04:21.721983 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.715664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:58193 (local address 127.20.121.130:39419)
0929 16:04:21.715965 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:21.715970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:21.717115 (+ 1145us) server_negotiation.cc:407] Connection header received
0929 16:04:21.717345 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:21.717349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:21.717434 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:21.717545 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:21.718056 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.718747 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.720162 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.720586 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.720751 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.721212 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:21.721322 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:21.721445 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:21.721586 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 16:04:21.721666 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260929 16:04:21.722136 23347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.715443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39419 (local address 127.20.121.131:58193)
0929 16:04:21.716983 (+ 1540us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:21.717001 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:21.717203 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:21.717562 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:21.717566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:21.717578 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:04:21.717884 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:21.717893 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.718893 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.718898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:21.720026 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:21.720037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.720784 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.720791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:21.720952 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.721014 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:04:21.721446 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:04:21.721459 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 16:04:21.721554 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 16:04:21.721694 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1314,"thread_start_us":1115,"threads_started":1}
I20260929 16:04:21.733253 22880 raft_consensus.cc:1261] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Refusing update from remote peer d537755f82cf42f4898789f370d67c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:21.733381 23019 raft_consensus.cc:1261] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Refusing update from remote peer d537755f82cf42f4898789f370d67c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:21.734143 23342 consensus_queue.cc:1035] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e8215f6e43b48ebb8e66a779f71f3fd" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:21.734640 23329 consensus_queue.cc:1035] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "95a95386def14a4fb180faf7d5132622" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:21.742874 23294 mvcc.cc:205] Tried to move back new op lower bound from 7334698441651433472 to 7334698441405857792. Current Snapshot: MvccSnapshot[applied={T|T < 7334698441651433472}]
I20260929 16:04:21.769408 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.762261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:36959 (local address 127.20.121.131:34383)
0929 16:04:21.762514 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:21.762521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:21.762894 (+ 373us) server_negotiation.cc:407] Connection header received
0929 16:04:21.763052 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:21.763056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:21.763132 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:21.763217 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:21.763887 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.765042 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.767389 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.767907 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.768061 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.768574 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:21.768756 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:21.768849 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:21.768968 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 16:04:21.769055 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 16:04:21.769534 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:21.762127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:34383 (local address 127.20.121.130:36959)
0929 16:04:21.762779 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:21.762798 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:21.762920 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:21.763293 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:21.763296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:21.763306 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:04:21.763690 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:21.763702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.765231 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.765240 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:21.767194 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:21.767220 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:21.768108 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:21.768117 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:21.768288 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:21.768344 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:04:21.768849 (+ 505us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:04:21.768865 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 16:04:21.768948 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 16:04:21.769110 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":365,"threads_started":1}
W20260929 16:04:21.770955 23294 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 16:04:23.384291 22721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:23.384511 22721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:23.389151 22721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fa2cefd26594a98a263fa38903325be: Sending DeleteTablet for 3 replicas of tablet 155ba6733fe34e8ba4f03310937ad5d9
I20260929 16:04:23.390089 22998 tablet_service.cc:1514] Processing DeleteTablet for tablet 155ba6733fe34e8ba4f03310937ad5d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:23 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:49320
I20260929 16:04:23.390188 22860 tablet_service.cc:1514] Processing DeleteTablet for tablet 155ba6733fe34e8ba4f03310937ad5d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:23 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:51188
I20260929 16:04:23.390259 23143 tablet_service.cc:1514] Processing DeleteTablet for tablet 155ba6733fe34e8ba4f03310937ad5d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:23 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:36110
I20260929 16:04:23.391155 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22794
I20260929 16:04:23.404052 23374 tablet_replica.cc:330] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622: stopping tablet replica
I20260929 16:04:23.404577 23376 tablet_replica.cc:330] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd: stopping tablet replica
I20260929 16:04:23.404563 23374 raft_consensus.cc:2229] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:23.404975 23374 raft_consensus.cc:2258] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:23.405002 23376 raft_consensus.cc:2229] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:23.405620 23376 raft_consensus.cc:2258] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:23.406247 23375 tablet_replica.cc:330] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a: stopping tablet replica
I20260929 16:04:23.406535 23375 raft_consensus.cc:2229] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:04:23.406804 23374 ts_tablet_manager.cc:1905] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:23.406804 23376 ts_tablet_manager.cc:1905] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:23.407290 23375 raft_consensus.cc:2258] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:23.408334 23375 ts_tablet_manager.cc:1905] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:23.414748 23375 ts_tablet_manager.cc:1918] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 16:04:23.414903 23375 log.cc:1196] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/155ba6733fe34e8ba4f03310937ad5d9
I20260929 16:04:23.414883 23374 ts_tablet_manager.cc:1918] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 16:04:23.414891 23376 ts_tablet_manager.cc:1918] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 16:04:23.415037 23374 log.cc:1196] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/155ba6733fe34e8ba4f03310937ad5d9
I20260929 16:04:23.415037 23376 log.cc:1196] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/155ba6733fe34e8ba4f03310937ad5d9
I20260929 16:04:23.415362 23375 ts_tablet_manager.cc:1939] T 155ba6733fe34e8ba4f03310937ad5d9 P d537755f82cf42f4898789f370d67c3a: Deleting consensus metadata
I20260929 16:04:23.415539 23376 ts_tablet_manager.cc:1939] T 155ba6733fe34e8ba4f03310937ad5d9 P 3e8215f6e43b48ebb8e66a779f71f3fd: Deleting consensus metadata
I20260929 16:04:23.415582 23374 ts_tablet_manager.cc:1939] T 155ba6733fe34e8ba4f03310937ad5d9 P 95a95386def14a4fb180faf7d5132622: Deleting consensus metadata
I20260929 16:04:23.416456 22707 catalog_manager.cc:4841] TS d537755f82cf42f4898789f370d67c3a (127.20.121.131:34383): tablet 155ba6733fe34e8ba4f03310937ad5d9 (table test-table [id=42a93af7d64a46caa45bc7ee581f1f9b]) successfully deleted
I20260929 16:04:23.416899 22707 catalog_manager.cc:4841] TS 3e8215f6e43b48ebb8e66a779f71f3fd (127.20.121.129:40363): tablet 155ba6733fe34e8ba4f03310937ad5d9 (table test-table [id=42a93af7d64a46caa45bc7ee581f1f9b]) successfully deleted
I20260929 16:04:23.416899 22708 catalog_manager.cc:4841] TS 95a95386def14a4fb180faf7d5132622 (127.20.121.130:39419): tablet 155ba6733fe34e8ba4f03310937ad5d9 (table test-table [id=42a93af7d64a46caa45bc7ee581f1f9b]) successfully deleted
Sep 29 16:04:23 dist-test-slave-x07d krb5kdc[22678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:23.428414 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:23.393547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.20.121.129:40363)
0929 16:04:23.404374 (+ 10827us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:23.404384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:23.404422 (+ 38us) server_negotiation.cc:407] Connection header received
0929 16:04:23.404526 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:23.404534 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:23.404661 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:23.404805 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:23.406877 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:23.408038 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:23.410065 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:23.410741 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:23.410873 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:23.422223 (+ 11350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:23.422241 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:23.422260 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:23.422309 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:23.425533 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:23.426400 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:23.426408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:23.426410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:23.426593 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:23.426977 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:23.426980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:23.426982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:23.427369 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:23.427622 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:23.427953 (+ 331us) server_negotiation.cc:299] Negotiation successful
0929 16:04:23.428137 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10571,"thread_start_us":10427,"threads_started":1}
W20260929 16:04:23.429219 22924 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51202
W20260929 16:04:23.429967 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22794: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:04:23.430295 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 22794
I20260929 16:04:23.457265 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22934
Sep 29 16:04:23 dist-test-slave-x07d krb5kdc[22678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:04:23.475114 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:23.458202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49344 (local address 127.20.121.130:39419)
0929 16:04:23.458849 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:23.458854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:23.458874 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:04:23.458934 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:23.458938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:23.459047 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:23.459281 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:23.460838 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:23.461696 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:23.463074 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:23.463467 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:23.463582 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:23.468357 (+ 4775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:23.468373 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:23.468390 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:23.468462 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:23.471896 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:23.472689 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:23.472699 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:23.472702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:23.472870 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:23.473311 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:23.473320 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:23.473323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:23.473777 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:23.474252 (+ 475us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:23.474449 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 16:04:23.474717 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":309,"threads_started":1}
W20260929 16:04:23.475857 23064 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49344
W20260929 16:04:23.476619 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22934: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:04:23.476820 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 22934
I20260929 16:04:23.503108 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 23073
Sep 29 16:04:23 dist-test-slave-x07d krb5kdc[22678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697861, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:04:23.521031 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:23.504008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.20.121.131:34383)
0929 16:04:23.504696 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:23.504701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:23.504720 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:04:23.504791 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:23.504796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:23.504868 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:23.504977 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:23.506447 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:23.507276 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:23.508518 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:23.508984 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:23.509133 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:23.514320 (+ 5187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:23.514338 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:23.514359 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:23.514408 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:23.517809 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:23.518678 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:23.518732 (+ 54us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:23.518735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:23.518902 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:23.519280 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:23.519283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:23.519285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:23.519765 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:23.520005 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:23.520359 (+ 354us) server_negotiation.cc:299] Negotiation successful
0929 16:04:23.520605 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":432,"threads_started":1}
W20260929 16:04:23.521874 23203 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36138
W20260929 16:04:23.522811 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 23073: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:04:23.523005 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 23073
I20260929 16:04:23.545217 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 22691
I20260929 16:04:23.546600 22763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:23.686690 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 22691
2026-09-29T16:04:23Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8541 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:23 dist-test-slave-x07d krb5kdc[23384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:23 dist-test-slave-x07d krb5kdc[23384](info): set up 2 sockets
Sep 29 16:04:23 dist-test-slave-x07d krb5kdc[23384](info): commencing operation
krb5kdc: starting...
W20260929 16:04:24.866408 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:24 dist-test-slave-x07d krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:04:25.070259 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37135
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:37135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:25.412014 23397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:25.412317 23397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:25.412485 23397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:25.419780 23397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:25.419893 23397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:25.419983 23397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:25.420065 23397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:25.420147 23397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:25.430246 23397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:37135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:25.431661 23397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:25.433029 23397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:25.445840 23403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:25.445911 23405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:25.445873 23402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:25.447553 23397 server_base.cc:1034] running on GCE node
I20260929 16:04:25.448508 23397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:25.450069 23397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:25.451316 23397 hybrid_clock.cc:648] HybridClock initialized: now 1790697865451263 us; error 51 us; skew 500 ppm
I20260929 16:04:25.451689 23397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:25 dist-test-slave-x07d krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697865, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:25.455220 23397 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:04:25.456970 23397 webserver.cc:460] Webserver started at http://127.20.121.190:45903/ using document root <none> and password file <none>
I20260929 16:04:25.457692 23397 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:25.457806 23397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:25.458235 23397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:25.461416 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "540d680ac8af4e31b05aa7607c8db8a1"
format_stamp: "Formatted at 2026-09-29 16:04:25 on dist-test-slave-x07d"
server_key: "4e9f53c96f75c65c9b3bf67cd5b8dee5"
server_key_iv: "5ae8e875cddb93847091be6883628758"
server_key_version: "encryptionkey@0"
I20260929 16:04:25.462273 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "540d680ac8af4e31b05aa7607c8db8a1"
format_stamp: "Formatted at 2026-09-29 16:04:25 on dist-test-slave-x07d"
server_key: "4e9f53c96f75c65c9b3bf67cd5b8dee5"
server_key_iv: "5ae8e875cddb93847091be6883628758"
server_key_version: "encryptionkey@0"
I20260929 16:04:25.467839 23397 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260929 16:04:25.471937 23412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:25.473583 23397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260929 16:04:25.473888 23397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "540d680ac8af4e31b05aa7607c8db8a1"
format_stamp: "Formatted at 2026-09-29 16:04:25 on dist-test-slave-x07d"
server_key: "4e9f53c96f75c65c9b3bf67cd5b8dee5"
server_key_iv: "5ae8e875cddb93847091be6883628758"
server_key_version: "encryptionkey@0"
I20260929 16:04:25.474190 23397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:25.491572 23397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:25.496486 23397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:25.496804 23397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:25.516788 23397 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:37135
I20260929 16:04:25.516793 23463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:37135 every 8 connection(s)
I20260929 16:04:25.518373 23397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:25.519709 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 23397
I20260929 16:04:25.519938 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:25.520577 20966 external_mini_cluster.cc:1428] Setting key 64b579e3455fec76b111dc56ff92f4cf
I20260929 16:04:25.523238 23464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 16:04:25 dist-test-slave-x07d krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:25.536830 23464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1: Bootstrap starting.
I20260929 16:04:25.540491 23464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:25.541683 23464 log.cc:824] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:25.545181 23464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1: No bootstrap required, opened a new log
I20260929 16:04:25.545964 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:25.522728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60674 (local address 127.20.121.190:37135)
0929 16:04:25.523612 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:25.523628 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:25.523682 (+ 54us) server_negotiation.cc:407] Connection header received
0929 16:04:25.526188 (+ 2506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:25.526218 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:25.526792 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:25.527269 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:25.528656 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:25.530832 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:25.532319 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:25.533476 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:25.533676 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:25.537992 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:25.538001 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:25.538086 (+ 85us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:25.538133 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:25.541375 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:25.542380 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:25.542394 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:25.542405 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:25.542648 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:25.543124 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:25.543129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:25.543132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:25.543690 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:25.543891 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:25.544409 (+ 518us) server_negotiation.cc:299] Negotiation successful
0929 16:04:25.544760 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":365,"threads_started":1}
I20260929 16:04:25.553309 23464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d680ac8af4e31b05aa7607c8db8a1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37135 } }
I20260929 16:04:25.553759 23464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:25.553884 23464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 540d680ac8af4e31b05aa7607c8db8a1, State: Initialized, Role: FOLLOWER
I20260929 16:04:25.554631 23464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d680ac8af4e31b05aa7607c8db8a1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37135 } }
I20260929 16:04:25.554931 23464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:25.555130 23464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:25.555338 23464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:25.557727 23464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d680ac8af4e31b05aa7607c8db8a1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37135 } }
I20260929 16:04:25.558331 23464 leader_election.cc:302] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 540d680ac8af4e31b05aa7607c8db8a1; no voters:
I20260929 16:04:25.558940 23464 leader_election.cc:288] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:25.559113 23469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:25.559876 23469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [term 1 LEADER]: Becoming Leader. State: Replica: 540d680ac8af4e31b05aa7607c8db8a1, State: Running, Role: LEADER
I20260929 16:04:25.560411 23469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d680ac8af4e31b05aa7607c8db8a1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37135 } }
I20260929 16:04:25.561291 23464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:25.563359 23470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "540d680ac8af4e31b05aa7607c8db8a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d680ac8af4e31b05aa7607c8db8a1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37135 } } }
I20260929 16:04:25.563591 23471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 540d680ac8af4e31b05aa7607c8db8a1. Latest consensus state: current_term: 1 leader_uuid: "540d680ac8af4e31b05aa7607c8db8a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d680ac8af4e31b05aa7607c8db8a1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37135 } } }
I20260929 16:04:25.564013 23470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:25.564055 23471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:25.565654 23477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:25.571604 23477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:25.581869 23477 catalog_manager.cc:1347] Generated new cluster ID: 0bf83592039b4b8aa0944ecddef1eeba
I20260929 16:04:25.582149 23477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:25.596233 23477 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:25.598318 23477 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:25.606143 23477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 540d680ac8af4e31b05aa7607c8db8a1: Generated new TSK 0
I20260929 16:04:25.607213 23477 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:04:25.662160 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37135
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:26.026337 23490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:26.026655 23490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:26.026751 23490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:26.034488 23490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:26.034621 23490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:26.034740 23490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:26.045279 23490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:26.046903 23490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:26.048352 23490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:26.062705 23498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:26.062768 23496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:26.063119 23495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:26.064185 23490 server_base.cc:1034] running on GCE node
I20260929 16:04:26.064965 23490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:26.066224 23490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:26.067476 23490 hybrid_clock.cc:648] HybridClock initialized: now 1790697866067412 us; error 73 us; skew 500 ppm
I20260929 16:04:26.067840 23490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:26 dist-test-slave-x07d krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697866, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:26.071130 23490 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:04:26.072710 23490 webserver.cc:460] Webserver started at http://127.20.121.129:38283/ using document root <none> and password file <none>
I20260929 16:04:26.073513 23490 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:26.073657 23490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:26.073971 23490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:26.076917 23490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "fe96a8031aea458c9d674ec48bf7728d"
format_stamp: "Formatted at 2026-09-29 16:04:26 on dist-test-slave-x07d"
server_key: "bc8faf3c76ba946a51e11044088acea4"
server_key_iv: "5082a0cf5f70f858cafdc1fc2b6ca66e"
server_key_version: "encryptionkey@0"
I20260929 16:04:26.078162 23490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "fe96a8031aea458c9d674ec48bf7728d"
format_stamp: "Formatted at 2026-09-29 16:04:26 on dist-test-slave-x07d"
server_key: "bc8faf3c76ba946a51e11044088acea4"
server_key_iv: "5082a0cf5f70f858cafdc1fc2b6ca66e"
server_key_version: "encryptionkey@0"
I20260929 16:04:26.083876 23490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 16:04:26.087643 23505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:26.089223 23490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260929 16:04:26.089408 23490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "fe96a8031aea458c9d674ec48bf7728d"
format_stamp: "Formatted at 2026-09-29 16:04:26 on dist-test-slave-x07d"
server_key: "bc8faf3c76ba946a51e11044088acea4"
server_key_iv: "5082a0cf5f70f858cafdc1fc2b6ca66e"
server_key_version: "encryptionkey@0"
I20260929 16:04:26.089637 23490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:26.117720 23490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:26.124795 23490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:26.125156 23490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:26.126540 23490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:26.128583 23490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:26.128679 23490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:26.128840 23490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:26.128909 23490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:26.166949 23490 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:36429
I20260929 16:04:26.167016 23617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:36429 every 8 connection(s)
I20260929 16:04:26.168282 23490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:26.175411 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 23490
I20260929 16:04:26.175647 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:26.176270 20966 external_mini_cluster.cc:1428] Setting key 96a585165c90be407bcb3a6e22a0e48e
Sep 29 16:04:26 dist-test-slave-x07d krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697866, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:26.189731 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:26.171013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:34809 (local address 127.20.121.190:37135)
0929 16:04:26.171457 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:26.171462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:26.172508 (+ 1046us) server_negotiation.cc:407] Connection header received
0929 16:04:26.174482 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:26.174488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:26.174581 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:26.174700 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:26.176303 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:26.177065 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:26.178747 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:26.179145 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:26.179283 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:26.183337 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:26.183346 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:26.183352 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:26.183398 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:26.186389 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:26.187374 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:26.187378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:26.187380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:26.187495 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:26.187989 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:26.187992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:26.187994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:26.188261 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:26.188381 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:26.189311 (+ 930us) server_negotiation.cc:299] Negotiation successful
0929 16:04:26.189474 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":213,"threads_started":1}
I20260929 16:04:26.191409 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:26.171604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37135 (local address 127.20.121.129:34809)
0929 16:04:26.172314 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:26.172357 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:26.174204 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:26.174943 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:26.174957 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:26.175468 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:26.176129 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:26.176151 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:26.177228 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:26.177233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:26.178607 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:26.178623 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:26.179371 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:26.179379 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:26.179644 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:26.180336 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:26.180360 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:26.183079 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:26.186537 (+ 3458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:26.186550 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:26.186571 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:26.187232 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:26.187609 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:26.187617 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:26.187623 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:26.187872 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:26.188389 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:26.188401 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:26.188889 (+ 488us) client_negotiation.cc:769] Sending connection context
0929 16:04:26.189219 (+ 330us) client_negotiation.cc:240] Negotiation successful
0929 16:04:26.189813 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":283,"threads_started":1}
I20260929 16:04:26.193406 23618 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37135
I20260929 16:04:26.193786 23618 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:26.194779 23618 heartbeater.cc:511] Master 127.20.121.190:37135 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:04:26.197034 23429 ts_manager.cc:194] Registered new tserver with Master: fe96a8031aea458c9d674ec48bf7728d (127.20.121.129:36429)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:04:26.198740 23429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:34809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:04:26.221961 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37135
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:26.575649 23624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:26.576015 23624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:26.576187 23624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:26.583949 23624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:26.584146 23624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:26.584344 23624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:26.595219 23624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:26.596669 23624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:26.598182 23624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:26.612562 23630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:26.612537 23632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:26.612818 23629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:26.613346 23624 server_base.cc:1034] running on GCE node
I20260929 16:04:26.614142 23624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:26.615195 23624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:26.616453 23624 hybrid_clock.cc:648] HybridClock initialized: now 1790697866616359 us; error 63 us; skew 500 ppm
I20260929 16:04:26.616894 23624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:26 dist-test-slave-x07d krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697866, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:26.620647 23624 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:04:26.622593 23624 webserver.cc:460] Webserver started at http://127.20.121.130:35311/ using document root <none> and password file <none>
I20260929 16:04:26.623365 23624 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:26.623476 23624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:26.623802 23624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:26.627077 23624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "c10af68df08148b5b95fbcd3fa93ebdb"
format_stamp: "Formatted at 2026-09-29 16:04:26 on dist-test-slave-x07d"
server_key: "00a48aa8af05ca55aa1de03676d3257d"
server_key_iv: "61a8024e4df60bfc5436cdea8adff3da"
server_key_version: "encryptionkey@0"
I20260929 16:04:26.627933 23624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "c10af68df08148b5b95fbcd3fa93ebdb"
format_stamp: "Formatted at 2026-09-29 16:04:26 on dist-test-slave-x07d"
server_key: "00a48aa8af05ca55aa1de03676d3257d"
server_key_iv: "61a8024e4df60bfc5436cdea8adff3da"
server_key_version: "encryptionkey@0"
I20260929 16:04:26.634088 23624 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260929 16:04:26.637912 23639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:26.639544 23624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 16:04:26.639750 23624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "c10af68df08148b5b95fbcd3fa93ebdb"
format_stamp: "Formatted at 2026-09-29 16:04:26 on dist-test-slave-x07d"
server_key: "00a48aa8af05ca55aa1de03676d3257d"
server_key_iv: "61a8024e4df60bfc5436cdea8adff3da"
server_key_version: "encryptionkey@0"
I20260929 16:04:26.639968 23624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:26.658824 23624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:26.664155 23624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:26.664479 23624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:26.665623 23624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:26.667248 23624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:26.667330 23624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:26.667470 23624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:26.667543 23624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:26.707181 23624 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:42013
I20260929 16:04:26.707296 23751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:42013 every 8 connection(s)
I20260929 16:04:26.708519 23624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:26.712973 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 23624
I20260929 16:04:26.713194 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:26.713608 20966 external_mini_cluster.cc:1428] Setting key 2a8ea082852fe07f8037ca1c5cf90f57
Sep 29 16:04:26 dist-test-slave-x07d krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697866, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:26.730496 23754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:26.711138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:44763 (local address 127.20.121.190:37135)
0929 16:04:26.711593 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:26.711597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:26.712604 (+ 1007us) server_negotiation.cc:407] Connection header received
0929 16:04:26.714924 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:26.714930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:26.715026 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:26.715145 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:26.717086 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:26.717810 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:26.719591 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:26.720048 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:26.720194 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:26.724485 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:26.724490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:26.724493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:26.724528 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:26.727164 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:26.727983 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:26.727995 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:26.727998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:26.728207 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:26.728679 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:26.728684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:26.728687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:26.728957 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:26.729112 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:26.729853 (+ 741us) server_negotiation.cc:299] Negotiation successful
0929 16:04:26.730127 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":252,"threads_started":1}
I20260929 16:04:26.731585 23755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:26.711640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37135 (local address 127.20.121.130:44763)
0929 16:04:26.712369 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:26.712485 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:26.714634 (+ 2149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:26.715449 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:26.715461 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:26.716161 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:26.716908 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:26.716928 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:26.717984 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:26.717989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:26.719430 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:26.719451 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:26.720296 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:26.720303 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:26.720555 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:26.721347 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:26.721389 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:26.724198 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:26.727341 (+ 3143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:26.727352 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:26.727372 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:26.727837 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:26.728349 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:26.728354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:26.728358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:26.728555 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:26.729095 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:26.729103 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:26.729481 (+ 378us) client_negotiation.cc:769] Sending connection context
0929 16:04:26.729766 (+ 285us) client_negotiation.cc:240] Negotiation successful
0929 16:04:26.730372 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":308,"threads_started":1}
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
I20260929 16:04:26.733242 23752 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37135
I20260929 16:04:26.733546 23752 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:26.734385 23752 heartbeater.cc:511] Master 127.20.121.190:37135 requested a full tablet report, sending...
I20260929 16:04:26.735882 23429 ts_manager.cc:194] Registered new tserver with Master: c10af68df08148b5b95fbcd3fa93ebdb (127.20.121.130:42013)
I20260929 16:04:26.736728 23429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:44763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
I20260929 16:04:27.202852 23618 heartbeater.cc:503] Master 127.20.121.190:37135 was elected leader, sending a full tablet report...
I20260929 16:04:27.739972 23752 heartbeater.cc:503] Master 127.20.121.190:37135 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
W20260929 16:04:27.903065 20966 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.20.121.131: real 1.186s user 0.000s sys 0.008s
I20260929 16:04:27.903869 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37135
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:28.247007 23758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:28.247308 23758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:28.247401 23758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:28.255214 23758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:28.255366 23758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:28.255601 23758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:28.266613 23758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:28.267992 23758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:28.269345 23758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:28.283353 23763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:28.283401 23764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:28.283414 23766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:28.284228 23758 server_base.cc:1034] running on GCE node
I20260929 16:04:28.284987 23758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:28.286123 23758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:28.287420 23758 hybrid_clock.cc:648] HybridClock initialized: now 1790697868287349 us; error 36 us; skew 500 ppm
I20260929 16:04:28.287781 23758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:28 dist-test-slave-x07d krb5kdc[23384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697868, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:28.291358 23758 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:04:28.292965 23758 webserver.cc:460] Webserver started at http://127.20.121.131:38569/ using document root <none> and password file <none>
I20260929 16:04:28.293792 23758 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:28.293967 23758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:28.294411 23758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:28.297710 23758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "12d62b18915847bd915635c4acbb287e"
format_stamp: "Formatted at 2026-09-29 16:04:28 on dist-test-slave-x07d"
server_key: "c952f30980895bae8b99876cfb8dec3c"
server_key_iv: "1f4f4ba2798097cdbae3acf99ab53d89"
server_key_version: "encryptionkey@0"
I20260929 16:04:28.298759 23758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "12d62b18915847bd915635c4acbb287e"
format_stamp: "Formatted at 2026-09-29 16:04:28 on dist-test-slave-x07d"
server_key: "c952f30980895bae8b99876cfb8dec3c"
server_key_iv: "1f4f4ba2798097cdbae3acf99ab53d89"
server_key_version: "encryptionkey@0"
I20260929 16:04:28.304603 23758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:04:28.308367 23773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:28.309863 23758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:04:28.310057 23758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "12d62b18915847bd915635c4acbb287e"
format_stamp: "Formatted at 2026-09-29 16:04:28 on dist-test-slave-x07d"
server_key: "c952f30980895bae8b99876cfb8dec3c"
server_key_iv: "1f4f4ba2798097cdbae3acf99ab53d89"
server_key_version: "encryptionkey@0"
I20260929 16:04:28.310281 23758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:28.323108 23758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:28.327514 23758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:28.327834 23758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:28.328820 23758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:28.330211 23758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:28.330283 23758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:28.330435 23758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:28.330487 23758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:28.366627 23758 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:38733
I20260929 16:04:28.366680 23885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:38733 every 8 connection(s)
I20260929 16:04:28.367980 23758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:28.373762 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 23758
I20260929 16:04:28.373937 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:28.374400 20966 external_mini_cluster.cc:1428] Setting key e378d923aaa37184a1b3ad46d1a7c616
Sep 29 16:04:28 dist-test-slave-x07d krb5kdc[23384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697868, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:28.390413 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:28.370870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:39873 (local address 127.20.121.190:37135)
0929 16:04:28.371467 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:28.371472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:28.372259 (+ 787us) server_negotiation.cc:407] Connection header received
0929 16:04:28.374444 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:28.374449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:28.374551 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:28.374678 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:28.377017 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:28.378121 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:28.379784 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:28.380193 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:28.380298 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:28.384103 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:28.384108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:28.384112 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:28.384144 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:28.386838 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:28.387602 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:28.387606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:28.387608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:28.387722 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:28.388412 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:28.388416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:28.388418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:28.388834 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:28.389024 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:28.389766 (+ 742us) server_negotiation.cc:299] Negotiation successful
0929 16:04:28.389989 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":339,"threads_started":1}
I20260929 16:04:28.391462 23889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:28.371371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37135 (local address 127.20.121.131:39873)
0929 16:04:28.372099 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:28.372138 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:28.374137 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:28.375007 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:28.375023 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:28.375866 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:28.376757 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:28.376784 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:28.378254 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:28.378258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:28.379640 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:28.379659 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:28.380418 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:28.380423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:28.380642 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:28.381395 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:28.381418 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:28.383857 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:28.386981 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:28.386988 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:28.387003 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:28.387465 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:28.387963 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:28.387968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:28.387971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:28.388264 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:28.388983 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:28.388992 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:28.389368 (+ 376us) client_negotiation.cc:769] Sending connection context
0929 16:04:28.389647 (+ 279us) client_negotiation.cc:240] Negotiation successful
0929 16:04:28.390276 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":280,"threads_started":1}
I20260929 16:04:28.392947 23886 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37135
I20260929 16:04:28.393293 23886 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:28.394186 23886 heartbeater.cc:511] Master 127.20.121.190:37135 requested a full tablet report, sending...
I20260929 16:04:28.395706 23429 ts_manager.cc:194] Registered new tserver with Master: 12d62b18915847bd915635c4acbb287e (127.20.121.131:38733)
I20260929 16:04:28.396471 23429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:39873
I20260929 16:04:28.403084 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 16:04:28.435674 23888 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:04:28.432752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36842 (local address 127.20.121.190:37135)
0929 16:04:28.433060 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:28.433069 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:28.433464 (+ 395us) server_negotiation.cc:407] Connection header received
0929 16:04:28.433728 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:28.433732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:28.433814 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:28.433930 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:28.435387 (+ 1457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36842: BlockingRecv error: recv got EOF from 127.0.0.1:36842 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 16:04:28.439059 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 23490
W20260929 16:04:28.442341 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 23490: Not authorized: Client connection negotiation failed: client connection to 127.20.121.129:36429: server requires authentication, but client does not have Kerberos credentials available
I20260929 16:04:28.442565 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 23490
I20260929 16:04:28.458619 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 23624
W20260929 16:04:28.461968 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 23624: Not authorized: Client connection negotiation failed: client connection to 127.20.121.130:42013: server requires authentication, but client does not have Kerberos credentials available
I20260929 16:04:28.462175 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 23624
I20260929 16:04:28.479557 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 23758
W20260929 16:04:28.482584 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 23758: Not authorized: Client connection negotiation failed: client connection to 127.20.121.131:38733: server requires authentication, but client does not have Kerberos credentials available
I20260929 16:04:28.482766 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 23758
I20260929 16:04:28.498238 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 23397
I20260929 16:04:28.499471 23459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:28.602713 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 23397
2026-09-29T16:04:28Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4913 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:28 dist-test-slave-x07d krb5kdc[23904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:28 dist-test-slave-x07d krb5kdc[23904](info): set up 2 sockets
Sep 29 16:04:28 dist-test-slave-x07d krb5kdc[23904](info): commencing operation
krb5kdc: starting...
W20260929 16:04:29.726871 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:29 dist-test-slave-x07d krb5kdc[23904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:04:29.934748 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33953
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:33953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:30.273748 23917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:30.274166 23917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:30.274333 23917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:30.281695 23917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:30.281813 23917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:30.281908 23917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:30.281998 23917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:30.282110 23917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:30.292142 23917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:33953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:30.293509 23917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:30.294896 23917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:30.307615 23923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:30.307659 23922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:30.307658 23925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:30.308565 23917 server_base.cc:1034] running on GCE node
I20260929 16:04:30.309409 23917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:30.311053 23917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:30.312292 23917 hybrid_clock.cc:648] HybridClock initialized: now 1790697870312230 us; error 51 us; skew 500 ppm
I20260929 16:04:30.312704 23917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:30 dist-test-slave-x07d krb5kdc[23904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:30.316011 23917 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:04:30.317505 23917 webserver.cc:460] Webserver started at http://127.20.121.190:40543/ using document root <none> and password file <none>
I20260929 16:04:30.318308 23917 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:30.318454 23917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:30.318770 23917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:30.321800 23917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "aef3ea5cc4c748bd823e4b7f224e8794"
format_stamp: "Formatted at 2026-09-29 16:04:30 on dist-test-slave-x07d"
server_key: "369175ff9506dcb2d7394f565e92f2f6"
server_key_iv: "3987998bd969914f29c53a3a768da208"
server_key_version: "encryptionkey@0"
I20260929 16:04:30.322669 23917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "aef3ea5cc4c748bd823e4b7f224e8794"
format_stamp: "Formatted at 2026-09-29 16:04:30 on dist-test-slave-x07d"
server_key: "369175ff9506dcb2d7394f565e92f2f6"
server_key_iv: "3987998bd969914f29c53a3a768da208"
server_key_version: "encryptionkey@0"
I20260929 16:04:30.327970 23917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:04:30.331462 23932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:30.332875 23917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:04:30.333060 23917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "aef3ea5cc4c748bd823e4b7f224e8794"
format_stamp: "Formatted at 2026-09-29 16:04:30 on dist-test-slave-x07d"
server_key: "369175ff9506dcb2d7394f565e92f2f6"
server_key_iv: "3987998bd969914f29c53a3a768da208"
server_key_version: "encryptionkey@0"
I20260929 16:04:30.333273 23917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:30.347055 23917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:30.351372 23917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:30.351667 23917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:30.370316 23917 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:33953
I20260929 16:04:30.370344 23983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:33953 every 8 connection(s)
I20260929 16:04:30.371711 23917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:30.375504 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 23917
I20260929 16:04:30.375559 23984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:30.375753 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:30.376379 20966 external_mini_cluster.cc:1428] Setting key 1cbb5fd5bf2cf698fd13657c74b8d8dc
I20260929 16:04:30.390177 23984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794: Bootstrap starting.
Sep 29 16:04:30 dist-test-slave-x07d krb5kdc[23904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:30.394181 23984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:30.395644 23984 log.cc:824] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:30.399181 23984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794: No bootstrap required, opened a new log
I20260929 16:04:30.401544 23987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:30.378262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.20.121.190:33953)
0929 16:04:30.379171 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:30.379188 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:30.379239 (+ 51us) server_negotiation.cc:407] Connection header received
0929 16:04:30.380955 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:30.380989 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:30.381555 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:30.382062 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:30.383453 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:30.385332 (+ 1879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:30.386717 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:30.387863 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:30.388069 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:30.392393 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:30.392404 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:30.392424 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:30.392479 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:30.396578 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:30.397465 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:30.397474 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:30.397482 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:30.397681 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:30.398215 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:30.398220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:30.398223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:30.398931 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:30.399274 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:30.399671 (+ 397us) server_negotiation.cc:299] Negotiation successful
0929 16:04:30.400159 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":367,"threads_started":1}
I20260929 16:04:30.405823 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:04:30.408390 23984 raft_consensus.cc:348] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef3ea5cc4c748bd823e4b7f224e8794" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33953 } }
I20260929 16:04:30.408792 23984 raft_consensus.cc:374] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:30.408921 23984 raft_consensus.cc:729] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aef3ea5cc4c748bd823e4b7f224e8794, State: Initialized, Role: FOLLOWER
I20260929 16:04:30.409567 23984 consensus_queue.cc:260] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef3ea5cc4c748bd823e4b7f224e8794" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33953 } }
I20260929 16:04:30.409801 23984 raft_consensus.cc:388] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:30.409924 23984 raft_consensus.cc:482] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:30.410169 23984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:30.412326 23984 raft_consensus.cc:504] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef3ea5cc4c748bd823e4b7f224e8794" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33953 } }
I20260929 16:04:30.412870 23984 leader_election.cc:302] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aef3ea5cc4c748bd823e4b7f224e8794; no voters:
I20260929 16:04:30.413530 23984 leader_election.cc:288] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:30.413623 23990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:30.415122 23990 raft_consensus.cc:686] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [term 1 LEADER]: Becoming Leader. State: Replica: aef3ea5cc4c748bd823e4b7f224e8794, State: Running, Role: LEADER
I20260929 16:04:30.415750 23990 consensus_queue.cc:237] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef3ea5cc4c748bd823e4b7f224e8794" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33953 } }
I20260929 16:04:30.416139 23984 sys_catalog.cc:564] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:30.418634 23992 sys_catalog.cc:455] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aef3ea5cc4c748bd823e4b7f224e8794. Latest consensus state: current_term: 1 leader_uuid: "aef3ea5cc4c748bd823e4b7f224e8794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef3ea5cc4c748bd823e4b7f224e8794" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33953 } } }
I20260929 16:04:30.419026 23992 sys_catalog.cc:458] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:30.419669 23991 sys_catalog.cc:455] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aef3ea5cc4c748bd823e4b7f224e8794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef3ea5cc4c748bd823e4b7f224e8794" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33953 } } }
I20260929 16:04:30.419974 23991 sys_catalog.cc:458] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:30.431702 23997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:30.437356 23997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:30.459245 23997 catalog_manager.cc:1347] Generated new cluster ID: 400388a309464217b6df2db826b808ce
I20260929 16:04:30.459532 23997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:30.477574 23987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:30.468485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41734 (local address 127.20.121.190:33953)
0929 16:04:30.468737 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:30.468748 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:30.470682 (+ 1934us) server_negotiation.cc:407] Connection header received
0929 16:04:30.470921 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:30.470926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:30.470993 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:30.471106 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:30.472206 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:30.472950 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:30.474308 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:30.474750 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:30.474902 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:30.476023 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:30.476028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:30.476031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:30.476732 (+ 701us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:30.476776 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:30.476804 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:30.476943 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:30.477081 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 16:04:30.477180 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 16:04:30.486167 23997 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:30.488095 23997 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:30.501457 23997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aef3ea5cc4c748bd823e4b7f224e8794: Generated new TSK 0
I20260929 16:04:30.502636 23997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:04:30.516731 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 23917
I20260929 16:04:30.518052 23979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:30.610944 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 23917
2026-09-29T16:04:30Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2001 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:30 dist-test-slave-x07d krb5kdc[24018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:30 dist-test-slave-x07d krb5kdc[24018](info): set up 2 sockets
Sep 29 16:04:30 dist-test-slave-x07d krb5kdc[24018](info): commencing operation
krb5kdc: starting...
W20260929 16:04:31.694591 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:31 dist-test-slave-x07d krb5kdc[24018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:04:31.906062 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33669
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:44713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:33669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:32.251785 24031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:32.252200 24031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:32.252372 24031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:32.259953 24031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:32.260144 24031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:32.260274 24031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:32.260382 24031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:32.260479 24031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:32.271224 24031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:44713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:33669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:32.272855 24031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:32.274438 24031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:32.287220 24036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:32.287124 24037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:32.287117 24039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:32.288421 24031 server_base.cc:1034] running on GCE node
I20260929 16:04:32.289309 24031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:32.290839 24031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:32.292044 24031 hybrid_clock.cc:648] HybridClock initialized: now 1790697872292003 us; error 30 us; skew 500 ppm
I20260929 16:04:32.292379 24031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:32 dist-test-slave-x07d krb5kdc[24018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697872, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:32.295755 24031 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:04:32.297511 24031 webserver.cc:460] Webserver started at http://127.20.121.190:34281/ using document root <none> and password file <none>
I20260929 16:04:32.298329 24031 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:32.298485 24031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:32.298847 24031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:32.302148 24031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "0c15bc74307148a8900479dda88ff7d8"
format_stamp: "Formatted at 2026-09-29 16:04:32 on dist-test-slave-x07d"
server_key: "36a62a7b4aefa54509b54dee56391225"
server_key_iv: "232de1a4b0baf89e3f1ed1078f33ab7d"
server_key_version: "encryptionkey@0"
I20260929 16:04:32.303010 24031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "0c15bc74307148a8900479dda88ff7d8"
format_stamp: "Formatted at 2026-09-29 16:04:32 on dist-test-slave-x07d"
server_key: "36a62a7b4aefa54509b54dee56391225"
server_key_iv: "232de1a4b0baf89e3f1ed1078f33ab7d"
server_key_version: "encryptionkey@0"
I20260929 16:04:32.309051 24031 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260929 16:04:32.312932 24046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:32.314565 24031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:32.314778 24031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "0c15bc74307148a8900479dda88ff7d8"
format_stamp: "Formatted at 2026-09-29 16:04:32 on dist-test-slave-x07d"
server_key: "36a62a7b4aefa54509b54dee56391225"
server_key_iv: "232de1a4b0baf89e3f1ed1078f33ab7d"
server_key_version: "encryptionkey@0"
I20260929 16:04:32.314987 24031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:32.343204 24031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:32.347519 24031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:32.347865 24031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:32.373070 24031 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:33669
I20260929 16:04:32.373162 24097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:33669 every 8 connection(s)
I20260929 16:04:32.374490 24031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:32.378207 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 24031
I20260929 16:04:32.378396 24098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:32.378438 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:32.379216 20966 external_mini_cluster.cc:1428] Setting key 1c8c005160c58f6f239f67c47c13380f
I20260929 16:04:32.392549 24098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8: Bootstrap starting.
Sep 29 16:04:32 dist-test-slave-x07d krb5kdc[24018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:32.396350 24098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:32.397594 24098 log.cc:824] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:32.400605 24098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8: No bootstrap required, opened a new log
I20260929 16:04:32.403287 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:32.381311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.20.121.190:33669)
0929 16:04:32.382209 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:32.382221 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:32.382267 (+ 46us) server_negotiation.cc:407] Connection header received
0929 16:04:32.384584 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:32.384610 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:32.385054 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:32.385456 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:32.387206 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:32.388461 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:32.390270 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:32.390917 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:32.391056 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:32.395916 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:32.395924 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:32.395940 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:32.395983 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:32.399207 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:32.399954 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:32.399960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:32.399967 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:32.400134 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:32.400521 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:32.400530 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:32.400533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:32.401099 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:32.401424 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:32.401789 (+ 365us) server_negotiation.cc:299] Negotiation successful
0929 16:04:32.402235 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":443,"threads_started":1}
I20260929 16:04:32.408609 24098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c15bc74307148a8900479dda88ff7d8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33669 } }
I20260929 16:04:32.409137 24098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:32.409289 24098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c15bc74307148a8900479dda88ff7d8, State: Initialized, Role: FOLLOWER
I20260929 16:04:32.409940 24098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c15bc74307148a8900479dda88ff7d8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33669 } }
I20260929 16:04:32.410236 24098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:32.410383 24098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:32.410583 24098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:32.413578 24098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c15bc74307148a8900479dda88ff7d8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33669 } }
I20260929 16:04:32.414223 24098 leader_election.cc:302] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c15bc74307148a8900479dda88ff7d8; no voters:
I20260929 16:04:32.414865 24098 leader_election.cc:288] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:32.414973 24103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:32.415748 24103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [term 1 LEADER]: Becoming Leader. State: Replica: 0c15bc74307148a8900479dda88ff7d8, State: Running, Role: LEADER
I20260929 16:04:32.416358 24103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c15bc74307148a8900479dda88ff7d8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33669 } }
I20260929 16:04:32.417181 24098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:32.419492 24104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c15bc74307148a8900479dda88ff7d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c15bc74307148a8900479dda88ff7d8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33669 } } }
I20260929 16:04:32.419863 24104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:32.420717 24109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:32.421324 24105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c15bc74307148a8900479dda88ff7d8. Latest consensus state: current_term: 1 leader_uuid: "0c15bc74307148a8900479dda88ff7d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c15bc74307148a8900479dda88ff7d8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33669 } } }
I20260929 16:04:32.421859 24105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:32.427435 24109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:32.439715 24109 catalog_manager.cc:1347] Generated new cluster ID: b6d3b99288634d9fa43452c0138bb2cc
I20260929 16:04:32.439859 24109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:32.465967 24109 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:32.467592 24109 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:32.476397 24109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c15bc74307148a8900479dda88ff7d8: Generated new TSK 0
I20260929 16:04:32.477324 24109 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:04:32.531026 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33669
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:44713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:32.881374 24124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:32.881744 24124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:32.881935 24124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:32.890194 24124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:32.890306 24124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:32.890419 24124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:32.900709 24124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:44713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:32.902061 24124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:32.903745 24124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:32.917735 24129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:32.917727 24130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:32.917688 24132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:32.918308 24124 server_base.cc:1034] running on GCE node
I20260929 16:04:32.919188 24124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:32.920262 24124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:32.921514 24124 hybrid_clock.cc:648] HybridClock initialized: now 1790697872921438 us; error 63 us; skew 500 ppm
I20260929 16:04:32.921995 24124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:32 dist-test-slave-x07d krb5kdc[24018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697872, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:32.925580 24124 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:04:32.927263 24124 webserver.cc:460] Webserver started at http://127.20.121.129:42477/ using document root <none> and password file <none>
I20260929 16:04:32.928015 24124 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:32.928179 24124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:32.928498 24124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:32.931695 24124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "9b42cf6e8c53485994e41d58a9386869"
format_stamp: "Formatted at 2026-09-29 16:04:32 on dist-test-slave-x07d"
server_key: "a74d38b07ef57a8fd0dfbd09d60a908f"
server_key_iv: "a9106de87e2cd254ed355057739db829"
server_key_version: "encryptionkey@0"
I20260929 16:04:32.932502 24124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "9b42cf6e8c53485994e41d58a9386869"
format_stamp: "Formatted at 2026-09-29 16:04:32 on dist-test-slave-x07d"
server_key: "a74d38b07ef57a8fd0dfbd09d60a908f"
server_key_iv: "a9106de87e2cd254ed355057739db829"
server_key_version: "encryptionkey@0"
I20260929 16:04:32.938131 24124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 16:04:32.941907 24139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:32.943504 24124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:32.943820 24124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "9b42cf6e8c53485994e41d58a9386869"
format_stamp: "Formatted at 2026-09-29 16:04:32 on dist-test-slave-x07d"
server_key: "a74d38b07ef57a8fd0dfbd09d60a908f"
server_key_iv: "a9106de87e2cd254ed355057739db829"
server_key_version: "encryptionkey@0"
I20260929 16:04:32.944038 24124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:32.961503 24124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:32.965731 24124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:32.966104 24124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:32.967172 24124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:32.968657 24124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:32.968768 24124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:32.968926 24124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:32.968999 24124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:33.003121 24124 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:46735
I20260929 16:04:33.003147 24251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:46735 every 8 connection(s)
I20260929 16:04:33.004732 24124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:33.011839 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 24124
I20260929 16:04:33.012079 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:33.012910 20966 external_mini_cluster.cc:1428] Setting key 8d67129a54df50a5faf59723fc20baa5
Sep 29 16:04:33 dist-test-slave-x07d krb5kdc[24018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697872, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:33.030506 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:33.007827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:54679 (local address 127.20.121.190:33669)
0929 16:04:33.008268 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:33.008272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:33.009485 (+ 1213us) server_negotiation.cc:407] Connection header received
0929 16:04:33.011383 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:33.011388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:33.011461 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:33.011561 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:33.013866 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:33.014686 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:33.017288 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:33.017953 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:33.018142 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:33.022935 (+ 4793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:33.022946 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:33.022955 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:33.023005 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:33.026970 (+ 3965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:33.027746 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:33.027755 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:33.027763 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:33.027926 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:33.028345 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:33.028354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:33.028361 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:33.028796 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:33.028962 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:33.029642 (+ 680us) server_negotiation.cc:299] Negotiation successful
0929 16:04:33.029911 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":229,"threads_started":1}
I20260929 16:04:33.031248 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:33.008511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33669 (local address 127.20.121.129:54679)
0929 16:04:33.009303 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:33.009342 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:33.011077 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:33.011898 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:33.011911 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:33.012758 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:33.013663 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:33.013686 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:33.014861 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:33.014866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:33.017055 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:33.017075 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:33.018241 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:33.018251 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:33.018619 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:33.019444 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:33.019479 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:33.022674 (+ 3195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:33.027131 (+ 4457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:33.027139 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:33.027154 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:33.027613 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:33.028047 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:33.028050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:33.028052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:33.028227 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:33.028924 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:33.028931 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:33.029266 (+ 335us) client_negotiation.cc:769] Sending connection context
0929 16:04:33.029490 (+ 224us) client_negotiation.cc:240] Negotiation successful
0929 16:04:33.029899 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":358,"threads_started":1}
I20260929 16:04:33.032954 24252 heartbeater.cc:347] Connected to a master server at 127.20.121.190:33669
I20260929 16:04:33.033358 24252 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:33.034443 24252 heartbeater.cc:511] Master 127.20.121.190:33669 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:04:33.036963 24063 ts_manager.cc:194] Registered new tserver with Master: 9b42cf6e8c53485994e41d58a9386869 (127.20.121.129:46735)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:04:33.038807 24063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:54679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:04:33.065943 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33669
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:44713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:33.431104 24258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:33.431428 24258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:33.431607 24258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:33.439574 24258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:33.439728 24258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:33.439990 24258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:33.450573 24258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:44713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:33.452054 24258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:33.453526 24258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:33.468577 24264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:33.468623 24263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:33.468600 24266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:33.469297 24258 server_base.cc:1034] running on GCE node
I20260929 16:04:33.469902 24258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:33.470932 24258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:33.472155 24258 hybrid_clock.cc:648] HybridClock initialized: now 1790697873472080 us; error 66 us; skew 500 ppm
I20260929 16:04:33.472500 24258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:33 dist-test-slave-x07d krb5kdc[24018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:33.476078 24258 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:04:33.477857 24258 webserver.cc:460] Webserver started at http://127.20.121.130:45349/ using document root <none> and password file <none>
I20260929 16:04:33.478645 24258 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:33.478760 24258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:33.479038 24258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:33.482203 24258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "9b23eed297ce412d982a6881b2c0f2be"
format_stamp: "Formatted at 2026-09-29 16:04:33 on dist-test-slave-x07d"
server_key: "11b862ac97dd9e0a205ba0a555966315"
server_key_iv: "a10d57849ed62574e59b4b5116a20304"
server_key_version: "encryptionkey@0"
I20260929 16:04:33.483043 24258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "9b23eed297ce412d982a6881b2c0f2be"
format_stamp: "Formatted at 2026-09-29 16:04:33 on dist-test-slave-x07d"
server_key: "11b862ac97dd9e0a205ba0a555966315"
server_key_iv: "a10d57849ed62574e59b4b5116a20304"
server_key_version: "encryptionkey@0"
I20260929 16:04:33.488938 24258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 16:04:33.492837 24273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:33.494504 24258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 16:04:33.494709 24258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "9b23eed297ce412d982a6881b2c0f2be"
format_stamp: "Formatted at 2026-09-29 16:04:33 on dist-test-slave-x07d"
server_key: "11b862ac97dd9e0a205ba0a555966315"
server_key_iv: "a10d57849ed62574e59b4b5116a20304"
server_key_version: "encryptionkey@0"
I20260929 16:04:33.494952 24258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:33.515937 24258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:33.521212 24258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:33.521638 24258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:33.522737 24258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:33.524089 24258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:33.524189 24258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:33.524340 24258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:33.524413 24258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:33.558307 24258 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:42609
I20260929 16:04:33.558349 24385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:42609 every 8 connection(s)
I20260929 16:04:33.559597 24258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:33.567000 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 24258
I20260929 16:04:33.567236 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:33.567785 20966 external_mini_cluster.cc:1428] Setting key 3b924886bdf7b4200a718a8f7fbc493f
Sep 29 16:04:33 dist-test-slave-x07d krb5kdc[24018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:33.583932 24388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:33.562231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:56073 (local address 127.20.121.190:33669)
0929 16:04:33.562661 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:33.562667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:33.563727 (+ 1060us) server_negotiation.cc:407] Connection header received
0929 16:04:33.565558 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:33.565563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:33.565632 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:33.565730 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:33.567818 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:33.568663 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:33.571946 (+ 3283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:33.572389 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:33.572488 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:33.577136 (+ 4648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:33.577142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:33.577146 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:33.577182 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:33.579921 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:33.580991 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:33.580995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:33.580998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:33.581157 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:33.581690 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:33.581694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:33.581696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:33.581968 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:33.582246 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:33.583402 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0929 16:04:33.583654 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":209,"threads_started":1}
I20260929 16:04:33.585608 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:33.562805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33669 (local address 127.20.121.130:56073)
0929 16:04:33.563556 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:33.563604 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:33.565282 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:33.565951 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:33.565959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:33.566723 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:33.567597 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:33.567625 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:33.569433 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:33.569438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:33.571752 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:33.571775 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:33.572653 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:33.572663 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:33.573023 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:33.573894 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:33.573939 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:33.576794 (+ 2855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:33.580100 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:33.580113 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:33.580137 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:33.580849 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:33.581276 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:33.581284 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:33.581290 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:33.581571 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:33.582254 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:33.582266 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:33.582806 (+ 540us) client_negotiation.cc:769] Sending connection context
0929 16:04:33.583204 (+ 398us) client_negotiation.cc:240] Negotiation successful
0929 16:04:33.583885 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":321,"threads_started":1}
I20260929 16:04:33.587394 24386 heartbeater.cc:347] Connected to a master server at 127.20.121.190:33669
I20260929 16:04:33.587769 24386 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
I20260929 16:04:33.588667 24386 heartbeater.cc:511] Master 127.20.121.190:33669 requested a full tablet report, sending...
I20260929 16:04:33.590348 24063 ts_manager.cc:194] Registered new tserver with Master: 9b23eed297ce412d982a6881b2c0f2be (127.20.121.130:42609)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
I20260929 16:04:33.591324 24063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:56073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:04:33.618978 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33669
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:44713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:33.960474 24392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:33.960816 24392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:33.960996 24392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:33.968269 24392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:33.968464 24392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:33.968663 24392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:33.978840 24392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:44713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:33.980278 24392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:33.981671 24392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:33.995319 24398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:33.995318 24397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:33.996088 24400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:33.996302 24392 server_base.cc:1034] running on GCE node
I20260929 16:04:33.997008 24392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:33.998158 24392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:33.999472 24392 hybrid_clock.cc:648] HybridClock initialized: now 1790697873999401 us; error 55 us; skew 500 ppm
I20260929 16:04:33.999925 24392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:34 dist-test-slave-x07d krb5kdc[24018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697874, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:34.010146 24392 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:04:34.011934 24392 webserver.cc:460] Webserver started at http://127.20.121.131:42269/ using document root <none> and password file <none>
I20260929 16:04:34.012687 24392 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:34.012801 24392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:34.013059 24392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:34.016242 24392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "c7968d1e96024571ad5cdb0ec60121b1"
format_stamp: "Formatted at 2026-09-29 16:04:34 on dist-test-slave-x07d"
server_key: "f768fb0dab2f96f5428765e8e91fa41d"
server_key_iv: "50d35639f56fa0c7c8df558331bb233b"
server_key_version: "encryptionkey@0"
I20260929 16:04:34.017131 24392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "c7968d1e96024571ad5cdb0ec60121b1"
format_stamp: "Formatted at 2026-09-29 16:04:34 on dist-test-slave-x07d"
server_key: "f768fb0dab2f96f5428765e8e91fa41d"
server_key_iv: "50d35639f56fa0c7c8df558331bb233b"
server_key_version: "encryptionkey@0"
I20260929 16:04:34.023064 24392 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 16:04:34.026816 24407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:34.028380 24392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:04:34.028520 24392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "c7968d1e96024571ad5cdb0ec60121b1"
format_stamp: "Formatted at 2026-09-29 16:04:34 on dist-test-slave-x07d"
server_key: "f768fb0dab2f96f5428765e8e91fa41d"
server_key_iv: "50d35639f56fa0c7c8df558331bb233b"
server_key_version: "encryptionkey@0"
I20260929 16:04:34.028687 24392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:34.042313 24252 heartbeater.cc:503] Master 127.20.121.190:33669 was elected leader, sending a full tablet report...
I20260929 16:04:34.052551 24392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:34.056915 24392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:34.057191 24392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:34.058182 24392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:34.059665 24392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:34.059744 24392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:34.059829 24392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:34.059872 24392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:34.094317 24392 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:38813
I20260929 16:04:34.094396 24519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:38813 every 8 connection(s)
I20260929 16:04:34.095674 24392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:34.100075 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 24392
I20260929 16:04:34.100301 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:34.100831 20966 external_mini_cluster.cc:1428] Setting key dd42d1278105bcdf68ad4fc2c3358e37
Sep 29 16:04:34 dist-test-slave-x07d krb5kdc[24018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697874, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:34.119906 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:34.098608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:42619 (local address 127.20.121.190:33669)
0929 16:04:34.099095 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:34.099100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:34.101439 (+ 2339us) server_negotiation.cc:407] Connection header received
0929 16:04:34.103096 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:34.103102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:34.103204 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:34.103318 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:34.105642 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.106486 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.108247 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.108666 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.108771 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:34.112803 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:34.112809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:34.112812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:34.112849 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:34.115936 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:34.116941 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:34.116947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:34.116951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:34.117095 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:34.117809 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:34.117816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:34.117820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:34.118203 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:34.118420 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:34.119221 (+ 801us) server_negotiation.cc:299] Negotiation successful
0929 16:04:34.119457 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":229,"threads_started":1}
I20260929 16:04:34.120954 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:34.099096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33669 (local address 127.20.121.131:42619)
0929 16:04:34.100060 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:34.100115 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:34.102639 (+ 2524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:34.103746 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:34.103758 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:34.104583 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:34.105413 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:34.105436 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.106647 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.106652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:34.108074 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:34.108089 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.108886 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.108891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:34.109167 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:34.109951 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:34.110147 (+ 196us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:34.112530 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:34.116111 (+ 3581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:34.116123 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:34.116144 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:34.116780 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:34.117352 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:34.117357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:34.117359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:34.117677 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:34.118368 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:34.118379 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:34.118851 (+ 472us) client_negotiation.cc:769] Sending connection context
0929 16:04:34.119148 (+ 297us) client_negotiation.cc:240] Negotiation successful
0929 16:04:34.119758 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":382,"threads_started":1}
I20260929 16:04:34.122573 24520 heartbeater.cc:347] Connected to a master server at 127.20.121.190:33669
I20260929 16:04:34.122946 24520 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:34.123668 24520 heartbeater.cc:511] Master 127.20.121.190:33669 requested a full tablet report, sending...
I20260929 16:04:34.125185 24063 ts_manager.cc:194] Registered new tserver with Master: c7968d1e96024571ad5cdb0ec60121b1 (127.20.121.131:38813)
I20260929 16:04:34.126097 24063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:42619
I20260929 16:04:34.130311 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 16:04:34 dist-test-slave-x07d krb5kdc[24018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697874, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 16:04:34 dist-test-slave-x07d krb5kdc[24018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697874, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:34.185823 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:34.170373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.20.121.190:33669)
0929 16:04:34.170548 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:34.170552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:34.171010 (+ 458us) server_negotiation.cc:407] Connection header received
0929 16:04:34.171161 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:34.171165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:34.171232 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:34.171322 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:34.173116 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.173874 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.175456 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.175994 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.176189 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:34.180551 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:34.180557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:34.180560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:34.180605 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:34.183271 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:34.183982 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:34.183986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:34.183988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:34.184132 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:34.184540 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:34.184543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:34.184545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:34.184861 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:34.185041 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:34.185419 (+ 378us) server_negotiation.cc:299] Negotiation successful
0929 16:04:34.185601 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 16:04:34.186597 24063 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59082
I20260929 16:04:34.188324 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24124
Sep 29 16:04:34 dist-test-slave-x07d krb5kdc[24018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697874, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:34.205078 24532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:34.189719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.20.121.129:46735)
0929 16:04:34.190375 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:34.190381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:34.190406 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:04:34.190461 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:34.190467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:34.190683 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:34.190806 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:34.192017 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.193004 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.194301 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.194880 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.194977 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:34.199114 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:34.199126 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:34.199141 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:34.199174 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:34.202288 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:34.202989 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:34.202995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:34.202998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:34.203116 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:34.203492 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:34.203501 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:34.203504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:34.203847 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:34.204158 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:34.204452 (+ 294us) server_negotiation.cc:299] Negotiation successful
0929 16:04:34.204728 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":348,"threads_started":1}
W20260929 16:04:34.206339 24247 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59436
W20260929 16:04:34.207417 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24124: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:04:34.207616 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 24124
I20260929 16:04:34.223784 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24258
Sep 29 16:04:34 dist-test-slave-x07d krb5kdc[24018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697874, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:04:34.240243 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:34.224617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.20.121.130:42609)
0929 16:04:34.225191 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:34.225198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:34.225224 (+ 26us) server_negotiation.cc:407] Connection header received
0929 16:04:34.225284 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:34.225290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:34.225505 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:34.225641 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:34.226880 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.227847 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.229101 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.229704 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.229818 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:34.234233 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:34.234246 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:34.234261 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:34.234295 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:34.237389 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:34.238213 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:34.238225 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:34.238231 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:34.238373 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:34.238740 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:34.238743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:34.238745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:34.239151 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:34.239384 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:34.239662 (+ 278us) server_negotiation.cc:299] Negotiation successful
0929 16:04:34.239890 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":259,"threads_started":1}
W20260929 16:04:34.241490 24381 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37488
W20260929 16:04:34.242738 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24258: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:04:34.242892 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 24258
I20260929 16:04:34.258865 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24392
Sep 29 16:04:34 dist-test-slave-x07d krb5kdc[24018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697874, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:04:34.275985 24534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:34.259882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.20.121.131:38813)
0929 16:04:34.260504 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:34.260511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:34.260539 (+ 28us) server_negotiation.cc:407] Connection header received
0929 16:04:34.260605 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:34.260611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:34.260823 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:34.260950 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:34.262330 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.263400 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.264684 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:34.265232 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:34.265330 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:34.269832 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:34.269847 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:34.269864 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:34.269908 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:34.273004 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:34.273738 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:34.273745 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:34.273747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:34.273876 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:34.274366 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:34.274370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:34.274372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:34.274784 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:34.275025 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:34.275342 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 16:04:34.275578 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":319,"threads_started":1}
W20260929 16:04:34.277400 24515 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41126
W20260929 16:04:34.278663 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24392: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:04:34.278805 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 24392
I20260929 16:04:34.294003 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24031
I20260929 16:04:34.295430 24093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:34.399564 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 24031
2026-09-29T16:04:34Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3793 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:34 dist-test-slave-x07d krb5kdc[24538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:34 dist-test-slave-x07d krb5kdc[24538](info): set up 2 sockets
Sep 29 16:04:34 dist-test-slave-x07d krb5kdc[24538](info): commencing operation
krb5kdc: starting...
W20260929 16:04:35.526598 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.011s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:35 dist-test-slave-x07d krb5kdc[24538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:04:35.735333 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34831
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:34831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:36.073632 24551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:36.074033 24551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:36.074218 24551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:36.082568 24551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:36.082707 24551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:36.082804 24551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:36.082886 24551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:36.082950 24551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:36.093071 24551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:34831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:36.094521 24551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:36.095873 24551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:36.108350 24559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:36.108413 24556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:36.108438 24557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:36.109236 24551 server_base.cc:1034] running on GCE node
I20260929 16:04:36.110066 24551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:36.111585 24551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:36.112838 24551 hybrid_clock.cc:648] HybridClock initialized: now 1790697876112793 us; error 45 us; skew 500 ppm
I20260929 16:04:36.113246 24551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:36 dist-test-slave-x07d krb5kdc[24538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697876, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:36.116572 24551 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:04:36.118255 24551 webserver.cc:460] Webserver started at http://127.20.121.190:42139/ using document root <none> and password file <none>
I20260929 16:04:36.118968 24551 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:36.119109 24551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:36.119412 24551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:36.122575 24551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde"
format_stamp: "Formatted at 2026-09-29 16:04:36 on dist-test-slave-x07d"
server_key: "0b906b26c531ca562d375160da92d5cc"
server_key_iv: "3c558af2a348c1c59357053ddd9d15c0"
server_key_version: "encryptionkey@0"
I20260929 16:04:36.123401 24551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde"
format_stamp: "Formatted at 2026-09-29 16:04:36 on dist-test-slave-x07d"
server_key: "0b906b26c531ca562d375160da92d5cc"
server_key_iv: "3c558af2a348c1c59357053ddd9d15c0"
server_key_version: "encryptionkey@0"
I20260929 16:04:36.128998 24551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260929 16:04:36.133404 24566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:36.135279 24551 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260929 16:04:36.135538 24551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde"
format_stamp: "Formatted at 2026-09-29 16:04:36 on dist-test-slave-x07d"
server_key: "0b906b26c531ca562d375160da92d5cc"
server_key_iv: "3c558af2a348c1c59357053ddd9d15c0"
server_key_version: "encryptionkey@0"
I20260929 16:04:36.135788 24551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:36.169052 24551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:36.175468 24551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:36.175853 24551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:36.200783 24617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:34831 every 8 connection(s)
I20260929 16:04:36.200834 24551 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:34831
I20260929 16:04:36.203284 24551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:36.206970 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 24551
I20260929 16:04:36.207193 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:36.208012 20966 external_mini_cluster.cc:1428] Setting key 21ba410cef1be07c071d7b4af0b8ffe6
I20260929 16:04:36.208294 24618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:36.223443 24618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde: Bootstrap starting.
Sep 29 16:04:36 dist-test-slave-x07d krb5kdc[24538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:36.227478 24618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:36.228559 24618 log.cc:824] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:36.231680 24618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde: No bootstrap required, opened a new log
I20260929 16:04:36.235468 24621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:36.210559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50586 (local address 127.20.121.190:34831)
0929 16:04:36.212745 (+ 2186us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:36.212772 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:36.212826 (+ 54us) server_negotiation.cc:407] Connection header received
0929 16:04:36.215327 (+ 2501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:36.215390 (+ 63us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:36.215907 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:36.216298 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:36.218174 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:36.219681 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:36.221518 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:36.222540 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:36.222704 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:36.227618 (+ 4914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:36.227631 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:36.227652 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:36.227705 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:36.231110 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:36.231844 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:36.231852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:36.231859 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:36.232020 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:36.232614 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:36.232624 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:36.232626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:36.233161 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:36.233509 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:36.233951 (+ 442us) server_negotiation.cc:299] Negotiation successful
0929 16:04:36.234436 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1904,"thread_start_us":1643,"threads_started":1}
I20260929 16:04:36.239750 24618 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34831 } }
I20260929 16:04:36.240120 24618 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:36.240232 24618 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e19cf78d6304c2b9ab2c26ebfc11bde, State: Initialized, Role: FOLLOWER
I20260929 16:04:36.240844 24618 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34831 } }
I20260929 16:04:36.241065 24618 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:36.241163 24618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:36.241284 24618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:36.243528 24618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34831 } }
I20260929 16:04:36.244048 24618 leader_election.cc:302] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e19cf78d6304c2b9ab2c26ebfc11bde; no voters:
I20260929 16:04:36.244777 24618 leader_election.cc:288] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:36.244908 24623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:36.245628 24623 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [term 1 LEADER]: Becoming Leader. State: Replica: 1e19cf78d6304c2b9ab2c26ebfc11bde, State: Running, Role: LEADER
I20260929 16:04:36.246222 24623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34831 } }
I20260929 16:04:36.247203 24618 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:36.249187 24625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e19cf78d6304c2b9ab2c26ebfc11bde. Latest consensus state: current_term: 1 leader_uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34831 } } }
I20260929 16:04:36.249328 24624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e19cf78d6304c2b9ab2c26ebfc11bde" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34831 } } }
I20260929 16:04:36.249595 24625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:36.249653 24624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:36.254565 24630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:36.260352 24630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:36.270289 24630 catalog_manager.cc:1347] Generated new cluster ID: a76583aa66f4469ab423500874de5ff7
I20260929 16:04:36.270447 24630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:36.287647 24630 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:36.289130 24630 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:36.298178 24630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e19cf78d6304c2b9ab2c26ebfc11bde: Generated new TSK 0
I20260929 16:04:36.299198 24630 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:04:36.359006 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34831
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:36.698917 24644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:36.699246 24644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:36.699342 24644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:36.706661 24644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:36.706779 24644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:36.706893 24644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:36.716851 24644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34831
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:36.718228 24644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:36.719676 24644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:36.733426 24650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:36.733409 24649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:36.733530 24652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:36.734642 24644 server_base.cc:1034] running on GCE node
I20260929 16:04:36.735376 24644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:36.736346 24644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:36.737548 24644 hybrid_clock.cc:648] HybridClock initialized: now 1790697876737499 us; error 40 us; skew 500 ppm
I20260929 16:04:36.737891 24644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:36 dist-test-slave-x07d krb5kdc[24538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697876, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:36.741178 24644 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:04:36.742830 24644 webserver.cc:460] Webserver started at http://127.20.121.129:40679/ using document root <none> and password file <none>
I20260929 16:04:36.743585 24644 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:36.743729 24644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:36.744055 24644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:36.747267 24644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "744b33a9dd4e48b49cf912f7ba01e5aa"
format_stamp: "Formatted at 2026-09-29 16:04:36 on dist-test-slave-x07d"
server_key: "f5ac458281f501f07a908af4c8a12cb7"
server_key_iv: "3920836e4811dfc6c6595d36a0a2e962"
server_key_version: "encryptionkey@0"
I20260929 16:04:36.748148 24644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "744b33a9dd4e48b49cf912f7ba01e5aa"
format_stamp: "Formatted at 2026-09-29 16:04:36 on dist-test-slave-x07d"
server_key: "f5ac458281f501f07a908af4c8a12cb7"
server_key_iv: "3920836e4811dfc6c6595d36a0a2e962"
server_key_version: "encryptionkey@0"
I20260929 16:04:36.753679 24644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260929 16:04:36.757385 24659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:36.758980 24644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:36.759171 24644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "744b33a9dd4e48b49cf912f7ba01e5aa"
format_stamp: "Formatted at 2026-09-29 16:04:36 on dist-test-slave-x07d"
server_key: "f5ac458281f501f07a908af4c8a12cb7"
server_key_iv: "3920836e4811dfc6c6595d36a0a2e962"
server_key_version: "encryptionkey@0"
I20260929 16:04:36.759392 24644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:36.780645 24644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:36.785352 24644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:36.785646 24644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:36.786681 24644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:36.788053 24644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:36.788122 24644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:36.788247 24644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:36.788311 24644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:36.822578 24644 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:34367
I20260929 16:04:36.822729 24771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:34367 every 8 connection(s)
I20260929 16:04:36.823938 24644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:36.836593 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 24644
I20260929 16:04:36.836793 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:36.837625 20966 external_mini_cluster.cc:1428] Setting key df866fa8abdf2bda50baa0dee28b069d
Sep 29 16:04:36 dist-test-slave-x07d krb5kdc[24538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697876, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:36.856577 24774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:36.826612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:55415 (local address 127.20.121.190:34831)
0929 16:04:36.827223 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:36.827229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:36.839703 (+ 12474us) server_negotiation.cc:407] Connection header received
0929 16:04:36.841956 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:36.841961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:36.842100 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:36.842209 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:36.843973 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:36.844708 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:36.846410 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:36.846804 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:36.846955 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:36.850641 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:36.850646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:36.850649 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:36.850679 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:36.853224 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:36.853984 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:36.853989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:36.853993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:36.854170 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:36.854673 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:36.854678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:36.854681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:36.855007 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:36.855162 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:36.855941 (+ 779us) server_negotiation.cc:299] Negotiation successful
0929 16:04:36.856165 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":345,"threads_started":1}
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:04:36.857719 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:36.827197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34831 (local address 127.20.121.129:55415)
0929 16:04:36.839482 (+ 12285us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:36.839550 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:36.841668 (+ 2118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:36.842641 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:36.842650 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:36.843207 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:36.843791 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:36.843809 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:36.844874 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:36.844880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:36.846253 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:36.846265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:36.847076 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:36.847084 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:36.847368 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:36.848112 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:36.848140 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:36.850396 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:36.853382 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:36.853392 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:36.853412 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:36.853847 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:36.854282 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:36.854288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:36.854291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:36.854535 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:36.855136 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:36.855144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:36.855556 (+ 412us) client_negotiation.cc:769] Sending connection context
0929 16:04:36.855835 (+ 279us) client_negotiation.cc:240] Negotiation successful
0929 16:04:36.856427 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11962,"thread_start_us":11723,"threads_started":1}
I20260929 16:04:36.859450 24772 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34831
I20260929 16:04:36.859809 24772 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:36.860558 24772 heartbeater.cc:511] Master 127.20.121.190:34831 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:04:36.862875 24583 ts_manager.cc:194] Registered new tserver with Master: 744b33a9dd4e48b49cf912f7ba01e5aa (127.20.121.129:34367)
I20260929 16:04:36.864355 24583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:55415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:04:36.890175 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34831
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:37.238420 24778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:37.238726 24778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:37.238914 24778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:37.246630 24778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:37.246773 24778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:37.247004 24778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:37.257251 24778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34831
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:37.258667 24778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:37.260076 24778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:37.274717 24783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:37.274742 24786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:37.274657 24784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:37.275339 24778 server_base.cc:1034] running on GCE node
I20260929 16:04:37.276068 24778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:37.277096 24778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:37.278312 24778 hybrid_clock.cc:648] HybridClock initialized: now 1790697877278270 us; error 39 us; skew 500 ppm
I20260929 16:04:37.278646 24778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:37 dist-test-slave-x07d krb5kdc[24538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697877, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:37.281805 24778 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:04:37.283434 24778 webserver.cc:460] Webserver started at http://127.20.121.130:41679/ using document root <none> and password file <none>
I20260929 16:04:37.284204 24778 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:37.284351 24778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:37.284701 24778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:37.287644 24778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "0c5148c7d383453fa0e3acd0c33a35ec"
format_stamp: "Formatted at 2026-09-29 16:04:37 on dist-test-slave-x07d"
server_key: "b5f3ac06abd8ffe7847efd474cd89e8e"
server_key_iv: "ab2bf46be8b43c79d4d3962e6fb4473b"
server_key_version: "encryptionkey@0"
I20260929 16:04:37.288410 24778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "0c5148c7d383453fa0e3acd0c33a35ec"
format_stamp: "Formatted at 2026-09-29 16:04:37 on dist-test-slave-x07d"
server_key: "b5f3ac06abd8ffe7847efd474cd89e8e"
server_key_iv: "ab2bf46be8b43c79d4d3962e6fb4473b"
server_key_version: "encryptionkey@0"
I20260929 16:04:37.293568 24778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 16:04:37.297181 24793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:37.298588 24778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:37.298753 24778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "0c5148c7d383453fa0e3acd0c33a35ec"
format_stamp: "Formatted at 2026-09-29 16:04:37 on dist-test-slave-x07d"
server_key: "b5f3ac06abd8ffe7847efd474cd89e8e"
server_key_iv: "ab2bf46be8b43c79d4d3962e6fb4473b"
server_key_version: "encryptionkey@0"
I20260929 16:04:37.298964 24778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:37.311488 24778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:37.315394 24778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:37.315685 24778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:37.316642 24778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:37.317938 24778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:37.318069 24778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:37.318228 24778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:37.318279 24778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:37.351398 24778 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:46067
I20260929 16:04:37.351438 24905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:46067 every 8 connection(s)
I20260929 16:04:37.352705 24778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:37.359812 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 24778
I20260929 16:04:37.359988 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:37.360411 20966 external_mini_cluster.cc:1428] Setting key 9fd9862c81f2d5cdae54d76d66f2b4a4
Sep 29 16:04:37 dist-test-slave-x07d krb5kdc[24538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697877, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:37.372982 24774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:37.355267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:48829 (local address 127.20.121.190:34831)
0929 16:04:37.355442 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:37.355446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:37.356677 (+ 1231us) server_negotiation.cc:407] Connection header received
0929 16:04:37.358628 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:37.358633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:37.358726 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:37.358837 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:37.360457 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.361201 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.362890 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.363323 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.363461 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:37.367224 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:37.367229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:37.367232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:37.367264 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:37.369858 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:37.370747 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:37.370753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:37.370756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:37.370879 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:37.371316 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:37.371321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:37.371324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:37.371587 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:37.371752 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:37.372429 (+ 677us) server_negotiation.cc:299] Negotiation successful
0929 16:04:37.372647 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 16:04:37.373998 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:37.355772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34831 (local address 127.20.121.130:48829)
0929 16:04:37.356516 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:37.356560 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:37.358316 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:37.359073 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:37.359081 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:37.359602 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:37.360290 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:37.360310 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.361365 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.361368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:37.362672 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:37.362688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.363559 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.363566 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:37.363811 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:37.364581 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:37.364602 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:37.366932 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:37.370164 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:37.370173 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:37.370191 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:37.370621 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:37.370986 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:37.370991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:37.370995 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:37.371196 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:37.371713 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:37.371722 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:37.372074 (+ 352us) client_negotiation.cc:769] Sending connection context
0929 16:04:37.372337 (+ 263us) client_negotiation.cc:240] Negotiation successful
0929 16:04:37.372884 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":302,"threads_started":1}
I20260929 16:04:37.375694 24906 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34831
I20260929 16:04:37.376075 24906 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:37.376801 24906 heartbeater.cc:511] Master 127.20.121.190:34831 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
I20260929 16:04:37.378358 24583 ts_manager.cc:194] Registered new tserver with Master: 0c5148c7d383453fa0e3acd0c33a35ec (127.20.121.130:46067)
I20260929 16:04:37.379153 24583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:48829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:04:37.405669 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34831
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:37.744186 24911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:37.744517 24911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:37.744684 24911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:37.752147 24911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:37.752324 24911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:37.752512 24911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:37.762632 24911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34831
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:37.764024 24911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:37.765357 24911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:37.779174 24917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:37.779147 24919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:37.779104 24916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:37.780045 24911 server_base.cc:1034] running on GCE node
I20260929 16:04:37.780663 24911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:37.781661 24911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:37.783021 24911 hybrid_clock.cc:648] HybridClock initialized: now 1790697877782948 us; error 46 us; skew 500 ppm
I20260929 16:04:37.783497 24911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:37 dist-test-slave-x07d krb5kdc[24538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697877, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:37.787230 24911 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:04:37.789136 24911 webserver.cc:460] Webserver started at http://127.20.121.131:41431/ using document root <none> and password file <none>
I20260929 16:04:37.789875 24911 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:37.790071 24911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:37.790407 24911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:37.793344 24911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "ace16c9313f54e92a84559c5629976d8"
format_stamp: "Formatted at 2026-09-29 16:04:37 on dist-test-slave-x07d"
server_key: "7cacd62a37d3dd58cf604c5ba4d4b4bd"
server_key_iv: "cbe5a720be54dd8863fe07c8e76d98db"
server_key_version: "encryptionkey@0"
I20260929 16:04:37.794204 24911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "ace16c9313f54e92a84559c5629976d8"
format_stamp: "Formatted at 2026-09-29 16:04:37 on dist-test-slave-x07d"
server_key: "7cacd62a37d3dd58cf604c5ba4d4b4bd"
server_key_iv: "cbe5a720be54dd8863fe07c8e76d98db"
server_key_version: "encryptionkey@0"
I20260929 16:04:37.799567 24911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260929 16:04:37.803938 24926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:37.805624 24911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:04:37.805850 24911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "ace16c9313f54e92a84559c5629976d8"
format_stamp: "Formatted at 2026-09-29 16:04:37 on dist-test-slave-x07d"
server_key: "7cacd62a37d3dd58cf604c5ba4d4b4bd"
server_key_iv: "cbe5a720be54dd8863fe07c8e76d98db"
server_key_version: "encryptionkey@0"
I20260929 16:04:37.806207 24911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:37.832144 24911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:37.836887 24911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:37.837248 24911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:37.838332 24911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:37.840224 24911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:37.840317 24911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:37.840453 24911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:37.840517 24911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:37.867540 24772 heartbeater.cc:503] Master 127.20.121.190:34831 was elected leader, sending a full tablet report...
I20260929 16:04:37.879106 24911 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:35475
I20260929 16:04:37.879204 25038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:35475 every 8 connection(s)
I20260929 16:04:37.880491 24911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:37.886162 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 24911
I20260929 16:04:37.886348 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:37.886811 20966 external_mini_cluster.cc:1428] Setting key 5686fc001df9f772e54a66718efe9e97
Sep 29 16:04:37 dist-test-slave-x07d krb5kdc[24538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697877, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:37.901677 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:37.883434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:50749 (local address 127.20.121.190:34831)
0929 16:04:37.883936 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:37.883941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:37.884880 (+ 939us) server_negotiation.cc:407] Connection header received
0929 16:04:37.887004 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:37.887011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:37.887115 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:37.887247 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:37.888994 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.889967 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.891578 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.892011 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.892168 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:37.895689 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:37.895693 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:37.895697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:37.895729 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:37.898271 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:37.899028 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:37.899034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:37.899037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:37.899167 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:37.899593 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:37.899599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:37.899602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:37.899880 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:37.900027 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:37.900900 (+ 873us) server_negotiation.cc:299] Negotiation successful
0929 16:04:37.901162 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":303,"threads_started":1}
I20260929 16:04:37.902743 25042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:37.883944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34831 (local address 127.20.121.131:50749)
0929 16:04:37.884722 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:37.884760 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:37.886692 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:37.887514 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:37.887526 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:37.888084 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:37.888828 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:37.888849 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.890146 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.890152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:37.891421 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:37.891433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.892257 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.892263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:37.892503 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:37.893205 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:37.893227 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:37.895420 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:37.898423 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:37.898432 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:37.898450 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:37.898894 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:37.899281 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:37.899286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:37.899290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:37.899475 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:37.900019 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:37.900027 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:37.900402 (+ 375us) client_negotiation.cc:769] Sending connection context
0929 16:04:37.900708 (+ 306us) client_negotiation.cc:240] Negotiation successful
0929 16:04:37.901253 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":333,"threads_started":1}
I20260929 16:04:37.904138 25039 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34831
I20260929 16:04:37.904484 25039 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:37.905272 25039 heartbeater.cc:511] Master 127.20.121.190:34831 requested a full tablet report, sending...
I20260929 16:04:37.906881 24583 ts_manager.cc:194] Registered new tserver with Master: ace16c9313f54e92a84559c5629976d8 (127.20.121.131:35475)
I20260929 16:04:37.907799 24583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:50749
I20260929 16:04:37.915613 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 16:04:37 dist-test-slave-x07d krb5kdc[24538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 16:04:37 dist-test-slave-x07d krb5kdc[24538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:37.966137 25047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:37.950515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42676 (local address 127.20.121.129:34367)
0929 16:04:37.951167 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:37.951175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:37.951202 (+ 27us) server_negotiation.cc:407] Connection header received
0929 16:04:37.951262 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:37.951269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:37.951482 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:37.951621 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:37.953275 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.954474 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.955813 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.956445 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.956547 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:37.960613 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:37.960625 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:37.960640 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:37.960674 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:37.963530 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:37.964191 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:37.964197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:37.964199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:37.964313 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:37.964666 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:37.964670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:37.964675 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:37.964991 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:37.965215 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:37.965442 (+ 227us) server_negotiation.cc:299] Negotiation successful
0929 16:04:37.965630 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":328,"threads_started":1}
Sep 29 16:04:37 dist-test-slave-x07d krb5kdc[24538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:37.985427 25041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:37.970911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50588 (local address 127.20.121.190:34831)
0929 16:04:37.971136 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:37.971142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:37.971271 (+ 129us) server_negotiation.cc:407] Connection header received
0929 16:04:37.971407 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:37.971410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:37.971472 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:37.971553 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:37.973016 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.973725 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.975164 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.975617 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.975736 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:37.979812 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:37.979817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:37.979820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:37.979856 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:37.982878 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:37.983552 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:37.983556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:37.983559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:37.983685 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:37.984156 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:37.984160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:37.984162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:37.984502 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:37.984680 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:37.984941 (+ 261us) server_negotiation.cc:299] Negotiation successful
0929 16:04:37.985119 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260929 16:04:37.986177 24583 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50588
I20260929 16:04:37.987331 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24644
I20260929 16:04:38.000576 25047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:37.988210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42688 (local address 127.20.121.129:34367)
0929 16:04:37.988445 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:37.988451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:37.988618 (+ 167us) server_negotiation.cc:407] Connection header received
0929 16:04:37.988734 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:37.988738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:37.988825 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:37.988905 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:37.990339 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.991086 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.992324 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:37.992734 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:37.992837 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:37.995449 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:37.995454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:37.995457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:37.995489 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:37.998124 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:37.998737 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:37.998741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:37.998743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:37.998855 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:37.999273 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:37.999277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:37.999286 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:37.999554 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:37.999739 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:38.000048 (+ 309us) server_negotiation.cc:299] Negotiation successful
0929 16:04:38.000214 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260929 16:04:38.001287 24767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:38.087543 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 24644
I20260929 16:04:38.104401 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24778
Sep 29 16:04:38 dist-test-slave-x07d krb5kdc[24538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:04:38.121843 25052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:38.105376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59164 (local address 127.20.121.130:46067)
0929 16:04:38.106055 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:38.106068 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:38.106108 (+ 40us) server_negotiation.cc:407] Connection header received
0929 16:04:38.106177 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:38.106185 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:38.106434 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:38.106580 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:38.107744 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:38.108777 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:38.110379 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:38.110933 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:38.111038 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:38.115587 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:38.115600 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:38.115617 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:38.115668 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:38.118954 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:38.119686 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:38.119693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:38.119695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:38.119830 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:38.120231 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:38.120234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:38.120236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:38.120574 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:38.120807 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:38.121316 (+ 509us) server_negotiation.cc:299] Negotiation successful
0929 16:04:38.121543 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":292,"threads_started":1}
I20260929 16:04:38.123271 24901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:38.215803 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 24778
I20260929 16:04:38.232200 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24911
Sep 29 16:04:38 dist-test-slave-x07d krb5kdc[24538](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:04:38.250239 25054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:38.233378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.20.121.131:35475)
0929 16:04:38.233987 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:38.233993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:38.234186 (+ 193us) server_negotiation.cc:407] Connection header received
0929 16:04:38.234270 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:38.234278 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:38.234499 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:38.234654 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:38.235843 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:38.236891 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:38.238337 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:38.238937 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:38.239042 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:38.243729 (+ 4687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:38.243743 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:38.243759 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:38.243803 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:38.246825 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:38.247684 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:38.247694 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:38.247696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:38.247851 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:38.248324 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:38.248330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:38.248333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:38.248813 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:38.249175 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:38.249482 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 16:04:38.249760 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":320,"threads_started":1}
I20260929 16:04:38.251669 25034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:38.336161 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 24911
I20260929 16:04:38.352202 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 24551
I20260929 16:04:38.353391 24613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:38.456621 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 24551
2026-09-29T16:04:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4057 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:38 dist-test-slave-x07d krb5kdc[25059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:38 dist-test-slave-x07d krb5kdc[25059](info): set up 2 sockets
Sep 29 16:04:38 dist-test-slave-x07d krb5kdc[25059](info): commencing operation
krb5kdc: starting...
W20260929 16:04:39.578503 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:39 dist-test-slave-x07d krb5kdc[25059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:04:39.798338 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:35817
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:35817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:40.135689 25072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:40.136018 25072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:40.136122 25072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:40.143501 25072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:40.143635 25072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:40.143683 25072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:40.143733 25072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:40.143782 25072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:40.158247 25072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:35817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:35817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:40.159600 25072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:40.160926 25072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:40.173756 25077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:40.173790 25078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:40.173954 25080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:40.174772 25072 server_base.cc:1034] running on GCE node
I20260929 16:04:40.175518 25072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:40.176937 25072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:40.178174 25072 hybrid_clock.cc:648] HybridClock initialized: now 1790697880178123 us; error 44 us; skew 500 ppm
I20260929 16:04:40.178535 25072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:40 dist-test-slave-x07d krb5kdc[25059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697880, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:40.188794 25072 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:04:40.190239 25072 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:40.190390 25072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:40.190753 25072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:40.193996 25072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "bc314b9fda13424a8a56e5d0c32c0303"
format_stamp: "Formatted at 2026-09-29 16:04:40 on dist-test-slave-x07d"
server_key: "ade13b1a55b8b5a10332a4101da15e39"
server_key_iv: "98cd1da5013b44c3c7bab3b976c7d307"
server_key_version: "encryptionkey@0"
I20260929 16:04:40.194895 25072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "bc314b9fda13424a8a56e5d0c32c0303"
format_stamp: "Formatted at 2026-09-29 16:04:40 on dist-test-slave-x07d"
server_key: "ade13b1a55b8b5a10332a4101da15e39"
server_key_iv: "98cd1da5013b44c3c7bab3b976c7d307"
server_key_version: "encryptionkey@0"
I20260929 16:04:40.200387 25072 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 16:04:40.204257 25086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:40.205821 25072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:04:40.206171 25072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "bc314b9fda13424a8a56e5d0c32c0303"
format_stamp: "Formatted at 2026-09-29 16:04:40 on dist-test-slave-x07d"
server_key: "ade13b1a55b8b5a10332a4101da15e39"
server_key_iv: "98cd1da5013b44c3c7bab3b976c7d307"
server_key_version: "encryptionkey@0"
I20260929 16:04:40.206444 25072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:40.222818 25072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:40.227655 25072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:40.227962 25072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:40.247876 25072 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:35817
I20260929 16:04:40.247897 25137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:35817 every 8 connection(s)
I20260929 16:04:40.248895 25072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:40.252803 25138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:40.259305 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 25072
I20260929 16:04:40.259521 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:40.260293 20966 external_mini_cluster.cc:1428] Setting key 87cb11307f929f8b29188e3a378b7413
I20260929 16:04:40.264994 25138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303: Bootstrap starting.
I20260929 16:04:40.269049 25138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:40.270550 25138 log.cc:824] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:40.274304 25138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303: No bootstrap required, opened a new log
Sep 29 16:04:40 dist-test-slave-x07d krb5kdc[25059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:40.283442 25138 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc314b9fda13424a8a56e5d0c32c0303" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35817 } }
I20260929 16:04:40.283929 25138 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:40.284057 25138 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc314b9fda13424a8a56e5d0c32c0303, State: Initialized, Role: FOLLOWER
I20260929 16:04:40.284722 25138 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc314b9fda13424a8a56e5d0c32c0303" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35817 } }
I20260929 16:04:40.284991 25138 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:40.285151 25138 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:40.285355 25138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:40.287792 25138 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc314b9fda13424a8a56e5d0c32c0303" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35817 } }
I20260929 16:04:40.288342 25138 leader_election.cc:302] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc314b9fda13424a8a56e5d0c32c0303; no voters:
I20260929 16:04:40.289062 25138 leader_election.cc:288] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:40.289176 25143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:40.290222 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:40.263039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38476 (local address 127.20.121.190:35817)
0929 16:04:40.264030 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:40.264047 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:40.266790 (+ 2743us) server_negotiation.cc:407] Connection header received
0929 16:04:40.269488 (+ 2698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:40.269545 (+ 57us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:40.270150 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:40.270688 (+ 538us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:40.272623 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:40.274584 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:40.275934 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:40.277061 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:40.277259 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:40.281627 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:40.281641 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:40.281669 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:40.281724 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:40.285240 (+ 3516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:40.286203 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:40.286211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:40.286220 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:40.286411 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:40.286814 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:40.286819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:40.286822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:40.287415 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:40.287697 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:40.288181 (+ 484us) server_negotiation.cc:299] Negotiation successful
0929 16:04:40.288721 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":432,"threads_started":1}
I20260929 16:04:40.290351 25143 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [term 1 LEADER]: Becoming Leader. State: Replica: bc314b9fda13424a8a56e5d0c32c0303, State: Running, Role: LEADER
I20260929 16:04:40.290934 25143 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc314b9fda13424a8a56e5d0c32c0303" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35817 } }
I20260929 16:04:40.291671 25138 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:40.294647 25144 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc314b9fda13424a8a56e5d0c32c0303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc314b9fda13424a8a56e5d0c32c0303" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35817 } } }
I20260929 16:04:40.295241 25144 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:40.295128 25145 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc314b9fda13424a8a56e5d0c32c0303. Latest consensus state: current_term: 1 leader_uuid: "bc314b9fda13424a8a56e5d0c32c0303" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc314b9fda13424a8a56e5d0c32c0303" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35817 } } }
I20260929 16:04:40.295555 25145 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303 [sys.catalog]: This master's current role is: LEADER
W20260929 16:04:40.303968 25159 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 16:04:40.304154 25159 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 16:04:40.304298 25151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:40.305918 25151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:40.314465 25151 catalog_manager.cc:1347] Generated new cluster ID: b7a81830aa2b43f7a72ff1d7cbe90dd2
I20260929 16:04:40.314641 25151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:40.335877 25151 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:40.337805 25151 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:40.354612 25151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303: Generated new TSK 0
I20260929 16:04:40.355779 25151 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:04:40.414783 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35817
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:40.759730 25164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:40.760093 25164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:40.760277 25164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:40.767681 25164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:40.767879 25164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:40.768064 25164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:40.778115 25164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:40.779567 25164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:40.780957 25164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:40.794719 25169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:40.794703 25170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:40.795353 25172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:40.795928 25164 server_base.cc:1034] running on GCE node
I20260929 16:04:40.796594 25164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:40.797721 25164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:40.798974 25164 hybrid_clock.cc:648] HybridClock initialized: now 1790697880798886 us; error 80 us; skew 500 ppm
I20260929 16:04:40.799386 25164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:40 dist-test-slave-x07d krb5kdc[25059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697880, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:40.802780 25164 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:04:40.804086 25164 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:40.804225 25164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:40.804561 25164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:40.807997 25164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "92831793cd3f4d33a1db0a572957e2c0"
format_stamp: "Formatted at 2026-09-29 16:04:40 on dist-test-slave-x07d"
server_key: "00aa98275688785152f2877dd6731b8d"
server_key_iv: "81466f4a937d2e38f157854689a8321e"
server_key_version: "encryptionkey@0"
I20260929 16:04:40.808904 25164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "92831793cd3f4d33a1db0a572957e2c0"
format_stamp: "Formatted at 2026-09-29 16:04:40 on dist-test-slave-x07d"
server_key: "00aa98275688785152f2877dd6731b8d"
server_key_iv: "81466f4a937d2e38f157854689a8321e"
server_key_version: "encryptionkey@0"
I20260929 16:04:40.814775 25164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 16:04:40.818424 25178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:40.820081 25164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 16:04:40.820286 25164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "92831793cd3f4d33a1db0a572957e2c0"
format_stamp: "Formatted at 2026-09-29 16:04:40 on dist-test-slave-x07d"
server_key: "00aa98275688785152f2877dd6731b8d"
server_key_iv: "81466f4a937d2e38f157854689a8321e"
server_key_version: "encryptionkey@0"
I20260929 16:04:40.820520 25164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:40.839648 25164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:40.844112 25164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:40.844419 25164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:40.845430 25164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:40.846962 25164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:40.847085 25164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:40.847223 25164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:40.847312 25164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:40.882601 25164 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:35041
I20260929 16:04:40.882655 25290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:35041 every 8 connection(s)
I20260929 16:04:40.883481 25164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:40.887985 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 25164
I20260929 16:04:40.888181 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:40.888809 20966 external_mini_cluster.cc:1428] Setting key 2a80b20d7ca2527b78d8ad57fc5931a7
Sep 29 16:04:40 dist-test-slave-x07d krb5kdc[25059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697880, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:40.906427 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:40.885932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:45727 (local address 127.20.121.190:35817)
0929 16:04:40.886386 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:40.886391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:40.887486 (+ 1095us) server_negotiation.cc:407] Connection header received
0929 16:04:40.889521 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:40.889526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:40.889615 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:40.889742 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:40.891335 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:40.892097 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:40.894046 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:40.894542 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:40.894676 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:40.898781 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:40.898791 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:40.898798 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:40.898846 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:40.902509 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:40.903453 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:40.903463 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:40.903469 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:40.903633 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:40.904144 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:40.904152 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:40.904159 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:40.904583 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:40.904749 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:40.905583 (+ 834us) server_negotiation.cc:299] Negotiation successful
0929 16:04:40.905887 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":236,"threads_started":1}
I20260929 16:04:40.907111 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:40.886539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:35817 (local address 127.20.121.129:45727)
0929 16:04:40.887324 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:40.887371 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:40.889245 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:40.889973 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:40.889982 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:40.890553 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:40.891166 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:40.891188 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:40.892292 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:40.892300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:40.893875 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:40.893887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:40.894764 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:40.894770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:40.895021 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:40.895783 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:40.895805 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:40.898521 (+ 2716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:40.902696 (+ 4175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:40.902704 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:40.902719 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:40.903286 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:40.903751 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:40.903754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:40.903757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:40.904006 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:40.904739 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:40.904745 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:40.905160 (+ 415us) client_negotiation.cc:769] Sending connection context
0929 16:04:40.905396 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 16:04:40.905826 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":395,"threads_started":1}
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:04:40.908941 25291 heartbeater.cc:347] Connected to a master server at 127.20.121.190:35817
I20260929 16:04:40.909261 25291 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:40.910135 25291 heartbeater.cc:511] Master 127.20.121.190:35817 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:04:40.912637 25103 ts_manager.cc:194] Registered new tserver with Master: 92831793cd3f4d33a1db0a572957e2c0 (127.20.121.129:35041)
I20260929 16:04:40.914865 25103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:45727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:04:40.938972 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35817
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:41.276863 25297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:41.277158 25297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:41.277256 25297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:41.284837 25297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:41.284961 25297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:41.285087 25297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:41.295503 25297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:41.296959 25297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:41.298457 25297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:41.312260 25302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:41.312167 25303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:41.312222 25305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:41.312987 25297 server_base.cc:1034] running on GCE node
I20260929 16:04:41.313622 25297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:41.314692 25297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:41.315968 25297 hybrid_clock.cc:648] HybridClock initialized: now 1790697881315867 us; error 66 us; skew 500 ppm
I20260929 16:04:41.316334 25297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:41 dist-test-slave-x07d krb5kdc[25059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697881, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:41.320658 25297 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:04:41.321965 25297 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:41.322147 25297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:41.322499 25297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:41.325969 25297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "1fa29ba2de2548ef8664f9a41388c022"
format_stamp: "Formatted at 2026-09-29 16:04:41 on dist-test-slave-x07d"
server_key: "863cb615947cbc95405017f86604e61f"
server_key_iv: "2c665d9fe8f83368bfadab6e99857ca7"
server_key_version: "encryptionkey@0"
I20260929 16:04:41.326859 25297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "1fa29ba2de2548ef8664f9a41388c022"
format_stamp: "Formatted at 2026-09-29 16:04:41 on dist-test-slave-x07d"
server_key: "863cb615947cbc95405017f86604e61f"
server_key_iv: "2c665d9fe8f83368bfadab6e99857ca7"
server_key_version: "encryptionkey@0"
I20260929 16:04:41.332321 25297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:04:41.336030 25311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:41.337543 25297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260929 16:04:41.337749 25297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "1fa29ba2de2548ef8664f9a41388c022"
format_stamp: "Formatted at 2026-09-29 16:04:41 on dist-test-slave-x07d"
server_key: "863cb615947cbc95405017f86604e61f"
server_key_iv: "2c665d9fe8f83368bfadab6e99857ca7"
server_key_version: "encryptionkey@0"
I20260929 16:04:41.337987 25297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:41.374727 25297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:41.379796 25297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:41.380100 25297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:41.381108 25297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:41.382529 25297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:41.382647 25297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:41.382789 25297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:41.382881 25297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:41.416175 25297 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:45085
I20260929 16:04:41.416263 25423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:45085 every 8 connection(s)
I20260929 16:04:41.417083 25297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:41.419626 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 25297
I20260929 16:04:41.419800 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:41.420359 20966 external_mini_cluster.cc:1428] Setting key ac169c3fbe5696bf6a7a3dd24c2ecc35
Sep 29 16:04:41 dist-test-slave-x07d krb5kdc[25059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697881, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
I20260929 16:04:41.440232 25426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:41.419719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:40153 (local address 127.20.121.190:35817)
0929 16:04:41.421370 (+ 1651us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:41.421377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:41.421839 (+ 462us) server_negotiation.cc:407] Connection header received
0929 16:04:41.423830 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:41.423836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:41.423951 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:41.424093 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:41.425856 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:41.426681 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:41.429039 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:41.429493 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:41.429616 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:41.434157 (+ 4541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:41.434163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:41.434166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:41.434200 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:41.437013 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:41.437814 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:41.437826 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:41.437828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:41.437940 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:41.438403 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:41.438409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:41.438413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:41.438703 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:41.438868 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:41.439620 (+ 752us) server_negotiation.cc:299] Negotiation successful
0929 16:04:41.439860 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1481,"thread_start_us":1342,"threads_started":1}
I20260929 16:04:41.441332 25427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:41.420414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:35817 (local address 127.20.121.130:40153)
0929 16:04:41.421693 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:41.421737 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:41.423555 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:41.424348 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:41.424361 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:41.424984 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:41.425661 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:41.425682 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:41.426850 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:41.426854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:41.428838 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:41.428858 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:41.429751 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:41.429761 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:41.430151 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:41.430976 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:41.431021 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:41.433836 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:41.437184 (+ 3348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:41.437194 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:41.437214 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:41.437672 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:41.438091 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:41.438096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:41.438099 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:41.438285 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:41.438839 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:41.438847 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:41.439221 (+ 374us) client_negotiation.cc:769] Sending connection context
0929 16:04:41.439497 (+ 276us) client_negotiation.cc:240] Negotiation successful
0929 16:04:41.440070 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"thread_start_us":747,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
I20260929 16:04:41.443110 25424 heartbeater.cc:347] Connected to a master server at 127.20.121.190:35817
I20260929 16:04:41.443441 25424 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:41.444130 25424 heartbeater.cc:511] Master 127.20.121.190:35817 requested a full tablet report, sending...
I20260929 16:04:41.445734 25103 ts_manager.cc:194] Registered new tserver with Master: 1fa29ba2de2548ef8664f9a41388c022 (127.20.121.130:45085)
I20260929 16:04:41.446632 25103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:40153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:04:41.470839 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35817
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:39637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:41.808691 25430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:41.809015 25430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:41.809208 25430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:41.816807 25430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:41.816938 25430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:41.817162 25430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:41.827778 25430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:41.829186 25430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:41.830530 25430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:41.844340 25438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:41.844509 25435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:41.845279 25436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:41.845381 25430 server_base.cc:1034] running on GCE node
I20260929 16:04:41.846113 25430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:41.847164 25430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:41.848402 25430 hybrid_clock.cc:648] HybridClock initialized: now 1790697881848347 us; error 45 us; skew 500 ppm
I20260929 16:04:41.848738 25430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:41 dist-test-slave-x07d krb5kdc[25059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697881, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:41.852240 25430 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:04:41.853439 25430 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:41.853534 25430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:41.853791 25430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:41.858141 25430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76"
format_stamp: "Formatted at 2026-09-29 16:04:41 on dist-test-slave-x07d"
server_key: "738bffa52144d6cbf662a12f98e6d33f"
server_key_iv: "91186452106d5d4d9f8629246620f4d0"
server_key_version: "encryptionkey@0"
I20260929 16:04:41.859282 25430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76"
format_stamp: "Formatted at 2026-09-29 16:04:41 on dist-test-slave-x07d"
server_key: "738bffa52144d6cbf662a12f98e6d33f"
server_key_iv: "91186452106d5d4d9f8629246620f4d0"
server_key_version: "encryptionkey@0"
I20260929 16:04:41.864992 25430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 16:04:41.868805 25444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:41.870494 25430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:41.870644 25430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76"
format_stamp: "Formatted at 2026-09-29 16:04:41 on dist-test-slave-x07d"
server_key: "738bffa52144d6cbf662a12f98e6d33f"
server_key_iv: "91186452106d5d4d9f8629246620f4d0"
server_key_version: "encryptionkey@0"
I20260929 16:04:41.870851 25430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:41.882803 25430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:41.887725 25430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:41.888059 25430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:41.889124 25430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:41.890779 25430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:41.890928 25430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:41.891064 25430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:41.891158 25430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:41.918833 25291 heartbeater.cc:503] Master 127.20.121.190:35817 was elected leader, sending a full tablet report...
I20260929 16:04:41.929713 25430 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:37941
I20260929 16:04:41.929819 25556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:37941 every 8 connection(s)
I20260929 16:04:41.930795 25430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:41.931178 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 25430
I20260929 16:04:41.931419 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:41.932180 20966 external_mini_cluster.cc:1428] Setting key 59a1d58f0b6efce1dc488b05b2ccf915
Sep 29 16:04:41 dist-test-slave-x07d krb5kdc[25059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697881, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:41.954144 25426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:41.934176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:36247 (local address 127.20.121.190:35817)
0929 16:04:41.934375 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:41.934379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:41.935583 (+ 1204us) server_negotiation.cc:407] Connection header received
0929 16:04:41.937692 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:41.937697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:41.937769 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:41.937860 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:41.939677 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:41.940507 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:41.942569 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:41.943045 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:41.943167 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:41.947201 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:41.947208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:41.947213 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:41.947265 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:41.950823 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:41.951658 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:41.951668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:41.951671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:41.951789 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:41.952266 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:41.952271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:41.952275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:41.952577 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:41.952746 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:41.953500 (+ 754us) server_negotiation.cc:299] Negotiation successful
0929 16:04:41.953741 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 16:04:41.955216 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:41.934413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:35817 (local address 127.20.121.131:36247)
0929 16:04:41.935385 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:41.935442 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:41.937389 (+ 1947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:41.938204 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:41.938214 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:41.938799 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:41.939432 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:41.939452 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:41.940674 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:41.940678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:41.942424 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:41.942437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:41.943270 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:41.943275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:41.943483 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:41.944296 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:41.944320 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:41.946964 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:41.950992 (+ 4028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:41.951003 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:41.951024 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:41.951506 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:41.951911 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:41.951916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:41.951920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:41.952139 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:41.952716 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:41.952725 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:41.953101 (+ 376us) client_negotiation.cc:769] Sending connection context
0929 16:04:41.953399 (+ 298us) client_negotiation.cc:240] Negotiation successful
0929 16:04:41.953985 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":396,"threads_started":1}
I20260929 16:04:41.956837 25557 heartbeater.cc:347] Connected to a master server at 127.20.121.190:35817
I20260929 16:04:41.957155 25557 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:41.957805 25557 heartbeater.cc:511] Master 127.20.121.190:35817 requested a full tablet report, sending...
I20260929 16:04:41.959539 25103 ts_manager.cc:194] Registered new tserver with Master: ea5523c8f39f42f3bc6d52f4a2d62a76 (127.20.121.131:37941)
I20260929 16:04:41.960812 25103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:36247
I20260929 16:04:41.961930 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:04:41.981769 25426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:41.967689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38490 (local address 127.20.121.190:35817)
0929 16:04:41.967948 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:41.967954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:41.968499 (+ 545us) server_negotiation.cc:407] Connection header received
0929 16:04:41.968638 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:41.968643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:41.968724 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:41.968819 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:41.970316 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:41.971331 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:41.972585 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:41.973057 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:41.973162 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:41.975726 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:41.975731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:41.975735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:41.975769 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:41.978906 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:41.979701 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:41.979706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:41.979708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:41.979858 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:41.980345 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:41.980349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:41.980351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:41.980685 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:41.980891 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:41.981299 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 16:04:41.981483 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260929 16:04:41.986675 25102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:04:41.994174 25102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:42.035089 25570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.008890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.20.121.131:37941)
0929 16:04:42.025591 (+ 16701us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:42.025599 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:42.025627 (+ 28us) server_negotiation.cc:407] Connection header received
0929 16:04:42.025727 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:42.025736 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:42.026069 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:42.027109 (+ 1040us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:42.027409 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.029104 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.032245 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.033765 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.033993 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.034214 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:42.034511 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 16:04:42.034706 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16515,"thread_start_us":16364,"threads_started":1}
I20260929 16:04:42.035142 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.008601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:37941 (local address 127.0.0.1:44052)
0929 16:04:42.023790 (+ 15189us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.023854 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:42.024879 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:42.026339 (+ 1460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:42.026349 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:42.026381 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:42.026896 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:42.026908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.029251 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.029257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:42.032033 (+ 2776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:42.032049 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.033995 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.034002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:42.034254 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.034306 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 16:04:42.034471 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 16:04:42.034721 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14768,"mutex_wait_us":1,"thread_start_us":15453,"threads_started":1}
I20260929 16:04:42.037003 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.009025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35041 (local address 127.0.0.1:51534)
0929 16:04:42.025330 (+ 16305us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.025351 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:42.025596 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:42.027819 (+ 2223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:42.027827 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:42.027854 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:42.028252 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:42.028263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.030198 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.030206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:42.033844 (+ 3638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:42.033863 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.036137 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.036143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:42.036309 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.036350 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 16:04:42.036444 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 16:04:42.036597 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16161,"mutex_wait_us":1324,"thread_start_us":15822,"threads_started":1}
I20260929 16:04:42.038249 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.009648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.20.121.129:35041)
0929 16:04:42.024686 (+ 15038us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:42.024696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:42.025550 (+ 854us) server_negotiation.cc:407] Connection header received
0929 16:04:42.025795 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:42.025801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:42.026069 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:42.027577 (+ 1508us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:42.028440 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.029985 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.034105 (+ 4120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.035908 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.036148 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.036341 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:42.036490 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 16:04:42.036660 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14652,"thread_start_us":14745,"threads_started":1}
I20260929 16:04:42.040228 25492 tablet_service.cc:1467] Processing CreateTablet for tablet 5e58906dc749405da9022939316e52c0 (DEFAULT_TABLE table=test-table [id=a1352178ce3d41d6bb5932d970b22d7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:42.042032 25492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e58906dc749405da9022939316e52c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:42.042313 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.025774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50076 (local address 127.20.121.130:45085)
0929 16:04:42.033128 (+ 7354us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:42.033137 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:42.033171 (+ 34us) server_negotiation.cc:407] Connection header received
0929 16:04:42.033382 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:42.033392 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:42.033689 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:42.033927 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:42.035089 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.036747 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.039430 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.040889 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.041064 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.041226 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:42.041818 (+ 592us) server_negotiation.cc:299] Negotiation successful
0929 16:04:42.041994 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7165,"thread_start_us":6377,"threads_started":1}
I20260929 16:04:42.042387 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.026112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:45085 (local address 127.0.0.1:50076)
0929 16:04:42.032988 (+ 6876us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.033009 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:42.033151 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:42.034270 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:42.034276 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:42.034305 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:42.034840 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:42.034858 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.036961 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.036967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:42.039239 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:42.039255 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.041372 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.041379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:42.041611 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.041648 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 16:04:42.041741 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 16:04:42.041887 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6710,"mutex_wait_us":67,"thread_start_us":6523,"threads_started":1}
I20260929 16:04:42.042106 25226 tablet_service.cc:1467] Processing CreateTablet for tablet 5e58906dc749405da9022939316e52c0 (DEFAULT_TABLE table=test-table [id=a1352178ce3d41d6bb5932d970b22d7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:42.043540 25226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e58906dc749405da9022939316e52c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:42.046406 25359 tablet_service.cc:1467] Processing CreateTablet for tablet 5e58906dc749405da9022939316e52c0 (DEFAULT_TABLE table=test-table [id=a1352178ce3d41d6bb5932d970b22d7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:42.047966 25359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e58906dc749405da9022939316e52c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:42.054482 25575 tablet_bootstrap.cc:492] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: Bootstrap starting.
I20260929 16:04:42.057664 25575 tablet_bootstrap.cc:654] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:42.058466 25576 tablet_bootstrap.cc:492] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: Bootstrap starting.
I20260929 16:04:42.059136 25575 log.cc:824] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:42.062574 25576 tablet_bootstrap.cc:654] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:42.063874 25576 log.cc:824] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:42.064164 25578 tablet_bootstrap.cc:492] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: Bootstrap starting.
I20260929 16:04:42.064895 25575 tablet_bootstrap.cc:492] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: No bootstrap required, opened a new log
I20260929 16:04:42.065177 25575 ts_tablet_manager.cc:1397] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260929 16:04:42.067160 25576 tablet_bootstrap.cc:492] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: No bootstrap required, opened a new log
I20260929 16:04:42.067420 25576 ts_tablet_manager.cc:1397] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260929 16:04:42.067638 25578 tablet_bootstrap.cc:654] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:42.068665 25578 log.cc:824] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:42.071329 25578 tablet_bootstrap.cc:492] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: No bootstrap required, opened a new log
I20260929 16:04:42.071614 25578 ts_tablet_manager.cc:1397] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260929 16:04:42.076454 25575 raft_consensus.cc:348] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 } } peers { permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 } } peers { permanent_uuid: "1fa29ba2de2548ef8664f9a41388c022" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45085 } }
I20260929 16:04:42.076946 25575 raft_consensus.cc:374] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:42.077145 25575 raft_consensus.cc:729] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92831793cd3f4d33a1db0a572957e2c0, State: Initialized, Role: FOLLOWER
I20260929 16:04:42.077857 25575 consensus_queue.cc:260] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 } } peers { permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 } } peers { permanent_uuid: "1fa29ba2de2548ef8664f9a41388c022" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45085 } }
I20260929 16:04:42.078027 25576 raft_consensus.cc:348] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 } } peers { permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 } } peers { permanent_uuid: "1fa29ba2de2548ef8664f9a41388c022" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45085 } }
I20260929 16:04:42.078572 25576 raft_consensus.cc:374] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:42.078708 25576 raft_consensus.cc:729] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea5523c8f39f42f3bc6d52f4a2d62a76, State: Initialized, Role: FOLLOWER
I20260929 16:04:42.079421 25576 consensus_queue.cc:260] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 } } peers { permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 } } peers { permanent_uuid: "1fa29ba2de2548ef8664f9a41388c022" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45085 } }
I20260929 16:04:42.079828 25575 ts_tablet_manager.cc:1428] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260929 16:04:42.080794 25557 heartbeater.cc:503] Master 127.20.121.190:35817 was elected leader, sending a full tablet report...
I20260929 16:04:42.080695 25578 raft_consensus.cc:348] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 } } peers { permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 } } peers { permanent_uuid: "1fa29ba2de2548ef8664f9a41388c022" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45085 } }
I20260929 16:04:42.081156 25578 raft_consensus.cc:374] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:42.081286 25578 raft_consensus.cc:729] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa29ba2de2548ef8664f9a41388c022, State: Initialized, Role: FOLLOWER
I20260929 16:04:42.081615 25576 ts_tablet_manager.cc:1428] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260929 16:04:42.081939 25578 consensus_queue.cc:260] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 } } peers { permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 } } peers { permanent_uuid: "1fa29ba2de2548ef8664f9a41388c022" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45085 } }
I20260929 16:04:42.083873 25424 heartbeater.cc:503] Master 127.20.121.190:35817 was elected leader, sending a full tablet report...
I20260929 16:04:42.084333 25578 ts_tablet_manager.cc:1428] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
W20260929 16:04:42.135845 25292 tablet.cc:2369] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:04:42.169171 25425 tablet.cc:2369] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:04:42.184033 25558 tablet.cc:2369] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:04:42.423310 25583 raft_consensus.cc:482] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:42.423679 25583 raft_consensus.cc:504] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 } } peers { permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 } } peers { permanent_uuid: "1fa29ba2de2548ef8664f9a41388c022" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45085 } }
I20260929 16:04:42.425906 25583 leader_election.cc:288] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea5523c8f39f42f3bc6d52f4a2d62a76 (127.20.121.131:37941), 92831793cd3f4d33a1db0a572957e2c0 (127.20.121.129:35041)
I20260929 16:04:42.435761 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.426799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:40073 (local address 127.20.121.129:35041)
0929 16:04:42.427170 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:42.427181 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:42.428300 (+ 1119us) server_negotiation.cc:407] Connection header received
0929 16:04:42.428611 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:42.428616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:42.428674 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:42.428749 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:42.429425 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.430537 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.433177 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.434609 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.434812 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.434896 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:42.435343 (+ 447us) server_negotiation.cc:299] Negotiation successful
0929 16:04:42.435472 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260929 16:04:42.435913 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.426482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35041 (local address 127.20.121.130:40073)
0929 16:04:42.428177 (+ 1695us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.428202 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:42.428352 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:42.428807 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:42.428813 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:42.428840 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:42.429262 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:42.429271 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.430676 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.430683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:42.432957 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:42.432974 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.434948 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.434955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:42.435129 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.435167 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 16:04:42.435286 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 16:04:42.435430 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1548,"mutex_wait_us":348,"thread_start_us":1000,"threads_started":1}
I20260929 16:04:42.436060 25570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.426565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:37927 (local address 127.20.121.131:37941)
0929 16:04:42.426746 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:42.426752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:42.427879 (+ 1127us) server_negotiation.cc:407] Connection header received
0929 16:04:42.428217 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:42.428222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:42.428302 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:42.428426 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:42.429127 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.430319 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.433177 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.434661 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.434827 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.434896 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:42.435575 (+ 679us) server_negotiation.cc:299] Negotiation successful
0929 16:04:42.435708 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 16:04:42.436007 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.426474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:37941 (local address 127.20.121.130:37927)
0929 16:04:42.427732 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.427758 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:42.428028 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:42.428450 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:42.428456 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:42.428484 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:42.428954 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:42.428973 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.430487 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.430491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:42.432962 (+ 2471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:42.432974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.435231 (+ 2257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.435241 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:42.435408 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.435435 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 16:04:42.435518 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 16:04:42.435646 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":57,"thread_start_us":869,"threads_started":1}
I20260929 16:04:42.437266 25512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e58906dc749405da9022939316e52c0" candidate_uuid: "1fa29ba2de2548ef8664f9a41388c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" is_pre_election: true
I20260929 16:04:42.437266 25246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e58906dc749405da9022939316e52c0" candidate_uuid: "1fa29ba2de2548ef8664f9a41388c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92831793cd3f4d33a1db0a572957e2c0" is_pre_election: true
I20260929 16:04:42.437767 25512 raft_consensus.cc:2454] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fa29ba2de2548ef8664f9a41388c022 in term 0.
I20260929 16:04:42.437757 25246 raft_consensus.cc:2454] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fa29ba2de2548ef8664f9a41388c022 in term 0.
I20260929 16:04:42.438565 25315 leader_election.cc:302] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa29ba2de2548ef8664f9a41388c022, 92831793cd3f4d33a1db0a572957e2c0; no voters:
I20260929 16:04:42.439095 25583 raft_consensus.cc:2781] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:42.439254 25583 raft_consensus.cc:482] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:42.439378 25583 raft_consensus.cc:3037] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:42.441807 25583 raft_consensus.cc:504] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 } } peers { permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 } } peers { permanent_uuid: "1fa29ba2de2548ef8664f9a41388c022" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45085 } }
I20260929 16:04:42.442562 25583 leader_election.cc:288] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [CANDIDATE]: Term 1 election: Requested vote from peers ea5523c8f39f42f3bc6d52f4a2d62a76 (127.20.121.131:37941), 92831793cd3f4d33a1db0a572957e2c0 (127.20.121.129:35041)
I20260929 16:04:42.443105 25512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e58906dc749405da9022939316e52c0" candidate_uuid: "1fa29ba2de2548ef8664f9a41388c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76"
I20260929 16:04:42.443181 25246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e58906dc749405da9022939316e52c0" candidate_uuid: "1fa29ba2de2548ef8664f9a41388c022" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92831793cd3f4d33a1db0a572957e2c0"
I20260929 16:04:42.443359 25512 raft_consensus.cc:3037] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:42.443428 25246 raft_consensus.cc:3037] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:42.445612 25246 raft_consensus.cc:2454] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fa29ba2de2548ef8664f9a41388c022 in term 1.
I20260929 16:04:42.445732 25512 raft_consensus.cc:2454] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fa29ba2de2548ef8664f9a41388c022 in term 1.
I20260929 16:04:42.446169 25315 leader_election.cc:302] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa29ba2de2548ef8664f9a41388c022, 92831793cd3f4d33a1db0a572957e2c0; no voters:
I20260929 16:04:42.446586 25583 raft_consensus.cc:2781] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:42.447139 25583 raft_consensus.cc:686] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 1 LEADER]: Becoming Leader. State: Replica: 1fa29ba2de2548ef8664f9a41388c022, State: Running, Role: LEADER
I20260929 16:04:42.447674 25583 consensus_queue.cc:237] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 } } peers { permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 } } peers { permanent_uuid: "1fa29ba2de2548ef8664f9a41388c022" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45085 } }
I20260929 16:04:42.452375 25101 catalog_manager.cc:5495] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fa29ba2de2548ef8664f9a41388c022 (127.20.121.130). New cstate: current_term: 1 leader_uuid: "1fa29ba2de2548ef8664f9a41388c022" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fa29ba2de2548ef8664f9a41388c022" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45085 } health_report { overall_health: HEALTHY } } }
I20260929 16:04:42.480413 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.474565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.20.121.130:45085)
0929 16:04:42.474846 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:42.474852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:42.474981 (+ 129us) server_negotiation.cc:407] Connection header received
0929 16:04:42.475323 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:42.475328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:42.475407 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:42.475596 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:04:42.475987 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.476710 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.478113 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.478519 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.478660 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.479198 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:04:42.479790 (+ 592us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:04:42.479972 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:42.480022 (+ 50us) server_negotiation.cc:299] Negotiation successful
0929 16:04:42.480119 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260929 16:04:42.492274 25512 raft_consensus.cc:1261] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76 [term 1 FOLLOWER]: Refusing update from remote peer 1fa29ba2de2548ef8664f9a41388c022: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:42.492274 25246 raft_consensus.cc:1261] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0 [term 1 FOLLOWER]: Refusing update from remote peer 1fa29ba2de2548ef8664f9a41388c022: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:42.493866 25586 consensus_queue.cc:1035] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92831793cd3f4d33a1db0a572957e2c0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:42.494563 25583 consensus_queue.cc:1035] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea5523c8f39f42f3bc6d52f4a2d62a76" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 37941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:42.531100 25103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:42.531517 25103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:42.536726 25103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc314b9fda13424a8a56e5d0c32c0303: Sending DeleteTablet for 3 replicas of tablet 5e58906dc749405da9022939316e52c0
I20260929 16:04:42.538488 25226 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e58906dc749405da9022939316e52c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:42 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:51534
I20260929 16:04:42.538488 25492 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e58906dc749405da9022939316e52c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:42 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:44052
I20260929 16:04:42.538586 25359 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e58906dc749405da9022939316e52c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:42 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:50076
I20260929 16:04:42.546805 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 25164
I20260929 16:04:42.551810 25599 tablet_replica.cc:330] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: stopping tablet replica
I20260929 16:04:42.553758 25599 raft_consensus.cc:2229] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:42.554859 25599 raft_consensus.cc:2258] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:42.555087 25601 tablet_replica.cc:330] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: stopping tablet replica
I20260929 16:04:42.555868 25601 raft_consensus.cc:2229] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:04:42.556986 25601 raft_consensus.cc:2258] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:42.558709 25600 tablet_replica.cc:330] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: stopping tablet replica
I20260929 16:04:42.559489 25600 raft_consensus.cc:2229] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:42.560144 25601 ts_tablet_manager.cc:1905] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:42.560159 25600 raft_consensus.cc:2258] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:42.560907 25599 ts_tablet_manager.cc:1905] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:42.568600 25601 ts_tablet_manager.cc:1918] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Sep 29 16:04:42 dist-test-slave-x07d krb5kdc[25059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:42.569010 25601 log.cc:1196] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/5e58906dc749405da9022939316e52c0
I20260929 16:04:42.569412 25600 ts_tablet_manager.cc:1905] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:42.569802 25601 ts_tablet_manager.cc:1939] T 5e58906dc749405da9022939316e52c0 P 1fa29ba2de2548ef8664f9a41388c022: Deleting consensus metadata
I20260929 16:04:42.572010 25090 catalog_manager.cc:4841] TS 1fa29ba2de2548ef8664f9a41388c022 (127.20.121.130:45085): tablet 5e58906dc749405da9022939316e52c0 (table test-table [id=a1352178ce3d41d6bb5932d970b22d7d]) successfully deleted
I20260929 16:04:42.578177 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.548848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51538 (local address 127.20.121.129:35041)
0929 16:04:42.549104 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:42.549113 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:42.553848 (+ 4735us) server_negotiation.cc:407] Connection header received
0929 16:04:42.554091 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:42.554100 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:42.554229 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:42.554359 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:42.560256 (+ 5897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.561334 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.563362 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.564105 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.564255 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.570238 (+ 5983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:42.570258 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:42.570280 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:42.570336 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:42.574261 (+ 3925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:42.575240 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:42.575251 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:42.575254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:42.575458 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:42.575913 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:42.575917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:42.575920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:42.576457 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:42.576780 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:42.577359 (+ 579us) server_negotiation.cc:299] Negotiation successful
0929 16:04:42.577639 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 16:04:42.579403 25599 ts_tablet_manager.cc:1918] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:04:42.579660 25286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:42.579821 25599 log.cc:1196] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/5e58906dc749405da9022939316e52c0
I20260929 16:04:42.580117 25600 ts_tablet_manager.cc:1918] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:04:42.580446 25600 log.cc:1196] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/5e58906dc749405da9022939316e52c0
I20260929 16:04:42.581393 25600 ts_tablet_manager.cc:1939] T 5e58906dc749405da9022939316e52c0 P ea5523c8f39f42f3bc6d52f4a2d62a76: Deleting consensus metadata
I20260929 16:04:42.584460 25089 catalog_manager.cc:4841] TS ea5523c8f39f42f3bc6d52f4a2d62a76 (127.20.121.131:37941): tablet 5e58906dc749405da9022939316e52c0 (table test-table [id=a1352178ce3d41d6bb5932d970b22d7d]) successfully deleted
I20260929 16:04:42.687291 25599 ts_tablet_manager.cc:1939] T 5e58906dc749405da9022939316e52c0 P 92831793cd3f4d33a1db0a572957e2c0: Deleting consensus metadata
I20260929 16:04:42.687930 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 25164
W20260929 16:04:42.706858 25090 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.121.129:35041 (error 108)
I20260929 16:04:42.707037 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 25297
I20260929 16:04:42.709303 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.708246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35041 (local address 127.0.0.1:51550)
0929 16:04:42.708910 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.709047 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:35041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":460,"threads_started":1}
W20260929 16:04:42.709585 25090 catalog_manager.cc:4584] TS 92831793cd3f4d33a1db0a572957e2c0 (127.20.121.129:35041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e58906dc749405da9022939316e52c0: Network error: Client connection negotiation failed: client connection to 127.20.121.129:35041: connect: Connection refused (error 111)
Sep 29 16:04:42 dist-test-slave-x07d krb5kdc[25059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:04:42.729826 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.708316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.20.121.130:45085)
0929 16:04:42.708561 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:42.708569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:42.708604 (+ 35us) server_negotiation.cc:407] Connection header received
0929 16:04:42.708712 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:42.708718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:42.708829 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:42.708982 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:42.711686 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.713246 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.716101 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.716613 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.716741 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.723026 (+ 6285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:42.723043 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:42.723062 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:42.723120 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:42.726712 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:42.727583 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:42.727592 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:42.727594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:42.727744 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:42.728161 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:42.728165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:42.728167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:42.728537 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:42.728910 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:42.729184 (+ 274us) server_negotiation.cc:299] Negotiation successful
0929 16:04:42.729409 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260929 16:04:42.730854 25419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:42.764115 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.763434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35041 (local address 127.0.0.1:51554)
0929 16:04:42.763754 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.763919 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:35041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260929 16:04:42.765498 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.765021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35041 (local address 127.0.0.1:51568)
0929 16:04:42.765226 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.765326 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:35041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 16:04:42.826817 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.826104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35041 (local address 127.0.0.1:51582)
0929 16:04:42.826440 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.826622 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:35041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260929 16:04:42.828260 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.827825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35041 (local address 127.0.0.1:51596)
0929 16:04:42.828003 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.828098 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:35041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 16:04:42.850325 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 25297
I20260929 16:04:42.871025 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 25430
Sep 29 16:04:42 dist-test-slave-x07d krb5kdc[25059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:04:42.887912 25570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.872040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.20.121.131:37941)
0929 16:04:42.872327 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:42.872333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:42.872356 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:04:42.872552 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:42.872558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:42.872630 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:42.872742 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:42.874190 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.874938 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.876259 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:42.876661 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:42.876804 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:42.881844 (+ 5040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:42.881861 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:42.881878 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:42.881923 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:42.885015 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:42.885758 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:42.885766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:42.885768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:42.885903 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:42.886324 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:42.886333 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:42.886336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:42.886684 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:42.886969 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:42.887300 (+ 331us) server_negotiation.cc:299] Negotiation successful
0929 16:04:42.887526 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260929 16:04:42.888897 25552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:42.909787 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.909077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35041 (local address 127.0.0.1:51602)
0929 16:04:42.909409 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.909545 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:35041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260929 16:04:42.911736 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:42.910954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35041 (local address 127.0.0.1:51614)
0929 16:04:42.911309 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:42.911457 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:35041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260929 16:04:42.994489 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 25430
I20260929 16:04:43.012044 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 25072
I20260929 16:04:43.013636 25133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:43.140373 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:43.139546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35041 (local address 127.0.0.1:51620)
0929 16:04:43.139846 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:43.140153 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:35041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260929 16:04:43.140815 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 25072
2026-09-29T16:04:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4691 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T16:04:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:43Z Disabled control of system clock
I20260929 16:04:43.223744 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:44703
--webserver_interface=127.20.121.190
--webserver_port=0
--builtin_ntp_servers=127.20.121.148:46167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:44703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 16:04:43.584774 25612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:43.585104 25612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:43.585191 25612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:43.592693 25612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:43.592823 25612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:43.592861 25612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:43.592911 25612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:43.592957 25612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:43.602840 25612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:46167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:44703
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:44703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:43.604192 25612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:43.606163 25612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:43.620826 25617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:43.620702 25618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:43.620680 25620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:43.621779 25612 server_base.cc:1034] running on GCE node
I20260929 16:04:43.622865 25612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:43.624672 25612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:43.625910 25612 hybrid_clock.cc:648] HybridClock initialized: now 1790697883625873 us; error 36 us; skew 500 ppm
I20260929 16:04:43.626431 25612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:04:43.629190 25612 webserver.cc:460] Webserver started at http://127.20.121.190:41239/ using document root <none> and password file <none>
I20260929 16:04:43.629997 25612 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:43.630182 25612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:43.630532 25612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:43.633749 25612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "1b06f8aad25042cca99ec7c764efedf8"
format_stamp: "Formatted at 2026-09-29 16:04:43 on dist-test-slave-x07d"
server_key: "621b513d66f43c41a92cb200566cd246"
server_key_iv: "a7a9b99a00ca2aaec0520de01c2683c4"
server_key_version: "encryptionkey@0"
I20260929 16:04:43.634737 25612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "1b06f8aad25042cca99ec7c764efedf8"
format_stamp: "Formatted at 2026-09-29 16:04:43 on dist-test-slave-x07d"
server_key: "621b513d66f43c41a92cb200566cd246"
server_key_iv: "a7a9b99a00ca2aaec0520de01c2683c4"
server_key_version: "encryptionkey@0"
I20260929 16:04:43.640853 25612 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260929 16:04:43.645294 25626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:43.647213 25612 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260929 16:04:43.647485 25612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "1b06f8aad25042cca99ec7c764efedf8"
format_stamp: "Formatted at 2026-09-29 16:04:43 on dist-test-slave-x07d"
server_key: "621b513d66f43c41a92cb200566cd246"
server_key_iv: "a7a9b99a00ca2aaec0520de01c2683c4"
server_key_version: "encryptionkey@0"
I20260929 16:04:43.647755 25612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:43.655143 25612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:43.656144 25612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:43.656476 25612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:43.680819 25612 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:44703
I20260929 16:04:43.680863 25677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:44703 every 8 connection(s)
I20260929 16:04:43.682384 25612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:43.686576 25678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:43.686985 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 25612
I20260929 16:04:43.687211 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:43.688006 20966 external_mini_cluster.cc:1428] Setting key 48317b174cde166b8306982a7c46f86c
I20260929 16:04:43.698164 25681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:43.690238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.20.121.190:44703)
0929 16:04:43.691479 (+ 1241us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:43.691491 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:43.691538 (+ 47us) server_negotiation.cc:407] Connection header received
0929 16:04:43.693619 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:43.693649 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:43.694345 (+ 696us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:43.694850 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:43.696078 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:43.696085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:43.696100 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:43.696115 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:43.696180 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:43.696198 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:43.696526 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:43.696721 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 16:04:43.697004 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":409,"threads_started":1}
I20260929 16:04:43.702033 25678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8: Bootstrap starting.
I20260929 16:04:43.705395 25678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:43.706698 25678 log.cc:824] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:43.710296 25678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8: No bootstrap required, opened a new log
I20260929 16:04:43.718277 25678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b06f8aad25042cca99ec7c764efedf8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44703 } }
I20260929 16:04:43.718724 25678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:43.718825 25678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b06f8aad25042cca99ec7c764efedf8, State: Initialized, Role: FOLLOWER
I20260929 16:04:43.719502 25678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b06f8aad25042cca99ec7c764efedf8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44703 } }
I20260929 16:04:43.719770 25678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:43.719918 25678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:43.720096 25678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:43.722512 25678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b06f8aad25042cca99ec7c764efedf8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44703 } }
I20260929 16:04:43.723091 25678 leader_election.cc:302] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b06f8aad25042cca99ec7c764efedf8; no voters:
I20260929 16:04:43.723737 25678 leader_election.cc:288] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:43.723908 25683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:43.724740 25683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [term 1 LEADER]: Becoming Leader. State: Replica: 1b06f8aad25042cca99ec7c764efedf8, State: Running, Role: LEADER
I20260929 16:04:43.725342 25683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b06f8aad25042cca99ec7c764efedf8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44703 } }
I20260929 16:04:43.726145 25678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:43.728363 25684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b06f8aad25042cca99ec7c764efedf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b06f8aad25042cca99ec7c764efedf8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44703 } } }
I20260929 16:04:43.728801 25684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:43.729161 25685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b06f8aad25042cca99ec7c764efedf8. Latest consensus state: current_term: 1 leader_uuid: "1b06f8aad25042cca99ec7c764efedf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b06f8aad25042cca99ec7c764efedf8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44703 } } }
I20260929 16:04:43.729632 25685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:43.733309 25691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:43.739296 25691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:43.748951 25691 catalog_manager.cc:1347] Generated new cluster ID: 82222808cec74f2f9028122d14eac655
I20260929 16:04:43.749081 25691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:43.771617 25691 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:43.772192 25691 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:43.786067 25691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8: Generated new TSK 0
I20260929 16:04:43.787086 25691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:04:43.802723 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44703
--builtin_ntp_servers=127.20.121.148:46167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 16:04:44.163110 25702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:44.163400 25702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:44.163489 25702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:44.171074 25702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:44.171202 25702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:44.171314 25702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:44.180811 25702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:46167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:44.182216 25702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:44.184015 25702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:44.198246 25707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:44.198318 25710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:44.198276 25708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:44.199777 25702 server_base.cc:1034] running on GCE node
I20260929 16:04:44.200570 25702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:44.201496 25702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:44.202711 25702 hybrid_clock.cc:648] HybridClock initialized: now 1790697884202646 us; error 55 us; skew 500 ppm
I20260929 16:04:44.203135 25702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:04:44.205657 25702 webserver.cc:460] Webserver started at http://127.20.121.129:35201/ using document root <none> and password file <none>
I20260929 16:04:44.206427 25702 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:44.206516 25702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:44.206795 25702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:44.209630 25702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "8491613c3ca74a5db113badfb73bbaf8"
format_stamp: "Formatted at 2026-09-29 16:04:44 on dist-test-slave-x07d"
server_key: "e6a7f0df552f18c873ecca2ed3eb60c4"
server_key_iv: "798b640c800ce0fe9d9aa3b123c0f6b3"
server_key_version: "encryptionkey@0"
I20260929 16:04:44.210528 25702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "8491613c3ca74a5db113badfb73bbaf8"
format_stamp: "Formatted at 2026-09-29 16:04:44 on dist-test-slave-x07d"
server_key: "e6a7f0df552f18c873ecca2ed3eb60c4"
server_key_iv: "798b640c800ce0fe9d9aa3b123c0f6b3"
server_key_version: "encryptionkey@0"
I20260929 16:04:44.215845 25702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 16:04:44.219420 25716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:44.220842 25702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:04:44.220990 25702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "8491613c3ca74a5db113badfb73bbaf8"
format_stamp: "Formatted at 2026-09-29 16:04:44 on dist-test-slave-x07d"
server_key: "e6a7f0df552f18c873ecca2ed3eb60c4"
server_key_iv: "798b640c800ce0fe9d9aa3b123c0f6b3"
server_key_version: "encryptionkey@0"
I20260929 16:04:44.221192 25702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:44.227275 25702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:44.228128 25702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:44.228402 25702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:44.229454 25702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:44.230986 25702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:44.231108 25702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:44.231235 25702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:44.231333 25702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:44.265239 25702 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:37415
I20260929 16:04:44.265347 25828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:37415 every 8 connection(s)
I20260929 16:04:44.266650 25702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:44.274628 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 25702
I20260929 16:04:44.274870 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:44.274945 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:44.269252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:60911 (local address 127.20.121.190:44703)
0929 16:04:44.269693 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:44.269700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:44.270920 (+ 1220us) server_negotiation.cc:407] Connection header received
0929 16:04:44.272561 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:44.272566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:44.272620 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:44.272719 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:44.274132 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:44.274139 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:44.274143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:44.274164 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:44.274215 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:44.274232 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:44.274375 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:44.274536 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 16:04:44.274605 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":216,"threads_started":1}
I20260929 16:04:44.275556 20966 external_mini_cluster.cc:1428] Setting key cc8ddaf57f0532e259c6e004f9c14aee
I20260929 16:04:44.275936 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:44.269844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:44703 (local address 127.20.121.129:60911)
0929 16:04:44.270753 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:44.270797 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:44.272299 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:44.272923 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:44.272932 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:44.273036 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:44.273748 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:04:44.273750 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:44.273788 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:04:44.273870 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:44.274337 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:44.274346 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:44.274351 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 16:04:44.274578 (+ 227us) client_negotiation.cc:240] Negotiation successful
0929 16:04:44.274836 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":306,"threads_started":1}
I20260929 16:04:44.277581 25829 heartbeater.cc:347] Connected to a master server at 127.20.121.190:44703
I20260929 16:04:44.277885 25829 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:44.278497 25829 heartbeater.cc:511] Master 127.20.121.190:44703 requested a full tablet report, sending...
I20260929 16:04:44.278522 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44703
--builtin_ntp_servers=127.20.121.148:46167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 16:04:44.280596 25643 ts_manager.cc:194] Registered new tserver with Master: 8491613c3ca74a5db113badfb73bbaf8 (127.20.121.129:37415)
W20260929 16:04:44.608448 25833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:44.608763 25833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:44.608855 25833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:44.616356 25833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:44.616479 25833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:44.616605 25833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:44.626292 25833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:46167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:44.627828 25833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:44.629608 25833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:44.644639 25838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:44.644582 25839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:44.645020 25841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:44.645614 25833 server_base.cc:1034] running on GCE node
I20260929 16:04:44.646368 25833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:44.647500 25833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:44.648777 25833 hybrid_clock.cc:648] HybridClock initialized: now 1790697884648720 us; error 53 us; skew 500 ppm
I20260929 16:04:44.649176 25833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:04:44.652060 25833 webserver.cc:460] Webserver started at http://127.20.121.130:32893/ using document root <none> and password file <none>
I20260929 16:04:44.652936 25833 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:44.653090 25833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:44.653445 25833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:44.656770 25833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "8d79dcb893ec46cb82dd16907a97e0e7"
format_stamp: "Formatted at 2026-09-29 16:04:44 on dist-test-slave-x07d"
server_key: "269d60a1b45248b7e37f6fa29109c2c9"
server_key_iv: "d6e97e2005394d5e7189a191456e8a65"
server_key_version: "encryptionkey@0"
I20260929 16:04:44.657707 25833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "8d79dcb893ec46cb82dd16907a97e0e7"
format_stamp: "Formatted at 2026-09-29 16:04:44 on dist-test-slave-x07d"
server_key: "269d60a1b45248b7e37f6fa29109c2c9"
server_key_iv: "d6e97e2005394d5e7189a191456e8a65"
server_key_version: "encryptionkey@0"
I20260929 16:04:44.663364 25833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 16:04:44.667466 25847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:44.669248 25833 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 16:04:44.669466 25833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "8d79dcb893ec46cb82dd16907a97e0e7"
format_stamp: "Formatted at 2026-09-29 16:04:44 on dist-test-slave-x07d"
server_key: "269d60a1b45248b7e37f6fa29109c2c9"
server_key_iv: "d6e97e2005394d5e7189a191456e8a65"
server_key_version: "encryptionkey@0"
I20260929 16:04:44.669731 25833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:44.675810 25833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:44.676702 25833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:44.676998 25833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:44.677930 25833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:44.679418 25833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:44.679551 25833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:44.679693 25833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:44.679783 25833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:44.717772 25833 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:43697
I20260929 16:04:44.717866 25959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:43697 every 8 connection(s)
I20260929 16:04:44.719221 25833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:44.728721 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:44.722203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:50869 (local address 127.20.121.190:44703)
0929 16:04:44.722540 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:44.722547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:44.724056 (+ 1509us) server_negotiation.cc:407] Connection header received
0929 16:04:44.726210 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:44.726215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:44.726288 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:44.726387 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:44.728036 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:44.728042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:44.728045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:44.728065 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:44.728100 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:44.728115 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:44.728230 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:44.728405 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 16:04:44.728459 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260929 16:04:44.729374 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 25833
I20260929 16:04:44.729579 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:44.729816 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:44.722780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:44703 (local address 127.20.121.130:50869)
0929 16:04:44.723843 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:44.723883 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:44.725737 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:44.726702 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:44.726711 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:44.726821 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:44.727675 (+ 854us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:04:44.727677 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:44.727734 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:04:44.727829 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:44.728216 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:44.728223 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:44.728229 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 16:04:44.728452 (+ 223us) client_negotiation.cc:240] Negotiation successful
0929 16:04:44.728704 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":320,"threads_started":1}
I20260929 16:04:44.730122 20966 external_mini_cluster.cc:1428] Setting key 0cb74a8b9e78629dc9554588bb23e8e3
I20260929 16:04:44.731571 25960 heartbeater.cc:347] Connected to a master server at 127.20.121.190:44703
I20260929 16:04:44.731921 25960 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:44.732620 25960 heartbeater.cc:511] Master 127.20.121.190:44703 requested a full tablet report, sending...
I20260929 16:04:44.733158 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44703
--builtin_ntp_servers=127.20.121.148:46167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 16:04:44.734454 25643 ts_manager.cc:194] Registered new tserver with Master: 8d79dcb893ec46cb82dd16907a97e0e7 (127.20.121.130:43697)
W20260929 16:04:45.091400 25963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:45.091725 25963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:45.091818 25963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:45.099438 25963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:45.099568 25963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:45.099700 25963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:45.109854 25963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:46167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:44703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:45.111407 25963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:45.113245 25963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:45.128521 25971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:45.128495 25969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:45.129202 25968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:45.129431 25963 server_base.cc:1034] running on GCE node
I20260929 16:04:45.130196 25963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:45.131366 25963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:45.132624 25963 hybrid_clock.cc:648] HybridClock initialized: now 1790697885132562 us; error 51 us; skew 500 ppm
I20260929 16:04:45.133101 25963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:04:45.136149 25963 webserver.cc:460] Webserver started at http://127.20.121.131:34887/ using document root <none> and password file <none>
I20260929 16:04:45.136991 25963 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:45.137091 25963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:45.137499 25963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:45.140725 25963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "fb516e6024c34e32a9b6fc0c7370d0f0"
format_stamp: "Formatted at 2026-09-29 16:04:45 on dist-test-slave-x07d"
server_key: "c049a842ed33526887b337922ca116b1"
server_key_iv: "b8b0bbd088730b15edbd0f4db8ee788f"
server_key_version: "encryptionkey@0"
I20260929 16:04:45.141680 25963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "fb516e6024c34e32a9b6fc0c7370d0f0"
format_stamp: "Formatted at 2026-09-29 16:04:45 on dist-test-slave-x07d"
server_key: "c049a842ed33526887b337922ca116b1"
server_key_iv: "b8b0bbd088730b15edbd0f4db8ee788f"
server_key_version: "encryptionkey@0"
I20260929 16:04:45.149573 25963 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260929 16:04:45.157059 25977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:45.159162 25963 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260929 16:04:45.159386 25963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "fb516e6024c34e32a9b6fc0c7370d0f0"
format_stamp: "Formatted at 2026-09-29 16:04:45 on dist-test-slave-x07d"
server_key: "c049a842ed33526887b337922ca116b1"
server_key_iv: "b8b0bbd088730b15edbd0f4db8ee788f"
server_key_version: "encryptionkey@0"
I20260929 16:04:45.159667 25963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:45.165948 25963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:45.167197 25963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:45.167523 25963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:45.168824 25963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:45.171974 25963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:45.172113 25963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:45.172235 25963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:45.172312 25963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:45.208395 25963 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:45775
I20260929 16:04:45.208510 26089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:45775 every 8 connection(s)
I20260929 16:04:45.209755 25963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:45.215829 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 25963
I20260929 16:04:45.216089 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:45.216626 20966 external_mini_cluster.cc:1428] Setting key ea638268c7197842ad991db8068b3c9b
I20260929 16:04:45.224754 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.212638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:33263 (local address 127.20.121.190:44703)
0929 16:04:45.212859 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.212865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.214540 (+ 1675us) server_negotiation.cc:407] Connection header received
0929 16:04:45.216380 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.216386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.216474 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.216592 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.223778 (+ 7186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.223784 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.223788 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.223812 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.223849 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.223867 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.224017 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.224285 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.224427 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 16:04:45.225920 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.213286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:44703 (local address 127.20.121.131:33263)
0929 16:04:45.214382 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.214421 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:45.216006 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:45.216862 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:45.216871 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:45.216975 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:45.223308 (+ 6333us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:04:45.223312 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:45.223372 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:04:45.223505 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:45.223990 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:45.224003 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:45.224012 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 16:04:45.224386 (+ 374us) client_negotiation.cc:240] Negotiation successful
0929 16:04:45.224747 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":309,"threads_started":1}
I20260929 16:04:45.227911 26090 heartbeater.cc:347] Connected to a master server at 127.20.121.190:44703
I20260929 16:04:45.228372 26090 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:45.229108 26090 heartbeater.cc:511] Master 127.20.121.190:44703 requested a full tablet report, sending...
I20260929 16:04:45.230960 25643 ts_manager.cc:194] Registered new tserver with Master: fb516e6024c34e32a9b6fc0c7370d0f0 (127.20.121.131:45775)
I20260929 16:04:45.232798 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:04:45.240983 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.238244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.20.121.190:44703)
0929 16:04:45.238471 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.238479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.238947 (+ 468us) server_negotiation.cc:407] Connection header received
0929 16:04:45.239173 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.239183 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.239238 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.239345 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.240346 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.240352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.240357 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.240376 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.240417 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.240430 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.240569 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.240643 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.240706 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 16:04:45.245750 25643 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:04:45.252879 25643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:04:45.283272 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.265939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:37415 (local address 127.0.0.1:45460)
0929 16:04:45.276314 (+ 10375us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.276376 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:45.276577 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:45.279777 (+ 3200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:45.279785 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:45.279883 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:45.280795 (+ 912us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:04:45.280800 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:45.280841 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:04:45.280930 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:45.282042 (+ 1112us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:45.282057 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:45.282065 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 16:04:45.282272 (+ 207us) client_negotiation.cc:240] Negotiation successful
0929 16:04:45.282384 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9950,"thread_start_us":12099,"threads_started":1}
I20260929 16:04:45.283515 25829 heartbeater.cc:503] Master 127.20.121.190:44703 was elected leader, sending a full tablet report...
I20260929 16:04:45.284431 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.266490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45460 (local address 127.20.121.129:37415)
0929 16:04:45.279233 (+ 12743us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.279239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.279263 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:04:45.279316 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.279321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.279476 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.280453 (+ 977us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.281808 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.281814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.281827 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.281838 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.281881 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.281895 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.282129 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.283943 (+ 1814us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.284057 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12584,"thread_start_us":12434,"threads_started":1}
I20260929 16:04:45.285154 26102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.267185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:45775 (local address 127.0.0.1:36362)
0929 16:04:45.277240 (+ 10055us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.277262 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:45.277365 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:45.282258 (+ 4893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:45.282263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:45.282289 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:45.283043 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:04:45.283046 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:45.283061 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:04:45.283113 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:45.284513 (+ 1400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:45.284518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:45.284523 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 16:04:45.284653 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 16:04:45.284707 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9868,"thread_start_us":9192,"threads_started":1}
I20260929 16:04:45.285988 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.266668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.20.121.131:45775)
0929 16:04:45.278129 (+ 11461us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.278143 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.278191 (+ 48us) server_negotiation.cc:407] Connection header received
0929 16:04:45.278283 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.278296 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.278621 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.282762 (+ 4141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.284262 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.284270 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.284286 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.284303 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.284357 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.284377 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.285440 (+ 1063us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.285517 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.285621 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11187,"thread_start_us":10977,"threads_started":1}
I20260929 16:04:45.288802 25764 tablet_service.cc:1467] Processing CreateTablet for tablet 7088b29302694d67b6bf56d4801d8045 (DEFAULT_TABLE table=test-table [id=7df152bc0ee24cadb149240035fa6bb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:45.290369 25764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7088b29302694d67b6bf56d4801d8045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:45.292990 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.278310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:43697 (local address 127.0.0.1:37446)
0929 16:04:45.283673 (+ 5363us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.283695 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:45.283785 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:45.290739 (+ 6954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:45.290746 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:45.290788 (+ 42us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:45.291461 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:04:45.291464 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:45.291484 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:04:45.291535 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:45.292566 (+ 1031us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:45.292571 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:45.292577 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 16:04:45.292658 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 16:04:45.292717 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5212,"thread_start_us":11269,"threads_started":1}
I20260929 16:04:45.293778 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.277807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.20.121.130:43697)
0929 16:04:45.290175 (+ 12368us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.290186 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.290220 (+ 34us) server_negotiation.cc:407] Connection header received
0929 16:04:45.290291 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.290299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.290538 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.290751 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.292286 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.292297 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.292313 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.292333 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.292388 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.292413 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.293266 (+ 853us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.293339 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.293446 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12182,"thread_start_us":12053,"threads_started":1}
I20260929 16:04:45.294126 26025 tablet_service.cc:1467] Processing CreateTablet for tablet 7088b29302694d67b6bf56d4801d8045 (DEFAULT_TABLE table=test-table [id=7df152bc0ee24cadb149240035fa6bb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:45.295681 26025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7088b29302694d67b6bf56d4801d8045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:45.298041 25895 tablet_service.cc:1467] Processing CreateTablet for tablet 7088b29302694d67b6bf56d4801d8045 (DEFAULT_TABLE table=test-table [id=7df152bc0ee24cadb149240035fa6bb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:04:45.299885 25895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7088b29302694d67b6bf56d4801d8045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:45.303421 26108 tablet_bootstrap.cc:492] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8: Bootstrap starting.
I20260929 16:04:45.307149 26108 tablet_bootstrap.cc:654] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:45.308307 26108 log.cc:824] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:45.311414 26110 tablet_bootstrap.cc:492] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: Bootstrap starting.
I20260929 16:04:45.312198 26108 tablet_bootstrap.cc:492] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8: No bootstrap required, opened a new log
I20260929 16:04:45.312510 26108 ts_tablet_manager.cc:1397] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 16:04:45.314715 26110 tablet_bootstrap.cc:654] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:45.315811 26110 log.cc:824] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:45.316988 26111 tablet_bootstrap.cc:492] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: Bootstrap starting.
I20260929 16:04:45.320961 26111 tablet_bootstrap.cc:654] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:45.322275 26111 log.cc:824] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:45.322489 26110 tablet_bootstrap.cc:492] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: No bootstrap required, opened a new log
I20260929 16:04:45.322249 26108 raft_consensus.cc:348] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 } } peers { permanent_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45775 } } peers { permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 } }
I20260929 16:04:45.322749 26108 raft_consensus.cc:374] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:45.322796 26110 ts_tablet_manager.cc:1397] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260929 16:04:45.322863 26108 raft_consensus.cc:729] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8491613c3ca74a5db113badfb73bbaf8, State: Initialized, Role: FOLLOWER
I20260929 16:04:45.323549 26108 consensus_queue.cc:260] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 } } peers { permanent_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45775 } } peers { permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 } }
I20260929 16:04:45.325639 26108 ts_tablet_manager.cc:1428] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260929 16:04:45.328928 26111 tablet_bootstrap.cc:492] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: No bootstrap required, opened a new log
I20260929 16:04:45.329267 26111 ts_tablet_manager.cc:1397] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260929 16:04:45.333726 26110 raft_consensus.cc:348] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 } } peers { permanent_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45775 } } peers { permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 } }
I20260929 16:04:45.334281 26110 raft_consensus.cc:374] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:45.334409 26110 raft_consensus.cc:729] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb516e6024c34e32a9b6fc0c7370d0f0, State: Initialized, Role: FOLLOWER
I20260929 16:04:45.335175 26110 consensus_queue.cc:260] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 } } peers { permanent_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45775 } } peers { permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 } }
I20260929 16:04:45.336473 26090 heartbeater.cc:503] Master 127.20.121.190:44703 was elected leader, sending a full tablet report...
I20260929 16:04:45.337160 26110 ts_tablet_manager.cc:1428] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260929 16:04:45.338519 26111 raft_consensus.cc:348] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 } } peers { permanent_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45775 } } peers { permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 } }
I20260929 16:04:45.338928 26111 raft_consensus.cc:374] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:45.339038 26111 raft_consensus.cc:729] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d79dcb893ec46cb82dd16907a97e0e7, State: Initialized, Role: FOLLOWER
I20260929 16:04:45.339764 26111 consensus_queue.cc:260] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 } } peers { permanent_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45775 } } peers { permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 } }
I20260929 16:04:45.340898 25960 heartbeater.cc:503] Master 127.20.121.190:44703 was elected leader, sending a full tablet report...
I20260929 16:04:45.341646 26111 ts_tablet_manager.cc:1428] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260929 16:04:45.387660 26115 raft_consensus.cc:482] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:04:45.387991 26115 raft_consensus.cc:504] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 } } peers { permanent_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45775 } } peers { permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 } }
I20260929 16:04:45.389892 26115 leader_election.cc:288] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8491613c3ca74a5db113badfb73bbaf8 (127.20.121.129:37415), 8d79dcb893ec46cb82dd16907a97e0e7 (127.20.121.130:43697)
I20260929 16:04:45.392840 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.390162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:37415 (local address 127.20.121.131:51797)
0929 16:04:45.390469 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.390489 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:45.390653 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:45.391134 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:45.391139 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:45.391166 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:45.391911 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:04:45.391914 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:45.391928 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:04:45.391968 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:45.392407 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:45.392411 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:45.392416 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 16:04:45.392535 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 16:04:45.392591 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260929 16:04:45.392882 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.390273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:51797 (local address 127.20.121.129:37415)
0929 16:04:45.390460 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.390467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.390787 (+ 320us) server_negotiation.cc:407] Connection header received
0929 16:04:45.390861 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.390868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.390932 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.391032 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.392234 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.392239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.392242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.392258 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.392291 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.392304 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.392446 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.392591 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.392650 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 16:04:45.393489 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.390660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:35931 (local address 127.20.121.130:43697)
0929 16:04:45.390843 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.390851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.391141 (+ 290us) server_negotiation.cc:407] Connection header received
0929 16:04:45.391272 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.391280 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.391345 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.391493 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.392764 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.392769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.392773 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.392791 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.392829 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.392844 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.392938 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.393170 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.393229 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 16:04:45.393901 25784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7088b29302694d67b6bf56d4801d8045" candidate_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8491613c3ca74a5db113badfb73bbaf8" is_pre_election: true
I20260929 16:04:45.394173 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.390304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:43697 (local address 127.20.121.131:35931)
0929 16:04:45.391032 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.391050 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:45.391149 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:45.391508 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:45.391513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:45.391536 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:45.392516 (+ 980us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:04:45.392522 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:45.392540 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:04:45.392598 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:45.393027 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:45.393034 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:45.393042 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 16:04:45.393712 (+ 670us) client_negotiation.cc:240] Negotiation successful
0929 16:04:45.393765 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":368,"threads_started":1}
I20260929 16:04:45.394577 25784 raft_consensus.cc:2454] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb516e6024c34e32a9b6fc0c7370d0f0 in term 0.
I20260929 16:04:45.395350 25978 leader_election.cc:302] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8491613c3ca74a5db113badfb73bbaf8, fb516e6024c34e32a9b6fc0c7370d0f0; no voters:
I20260929 16:04:45.395375 25915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7088b29302694d67b6bf56d4801d8045" candidate_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" is_pre_election: true
I20260929 16:04:45.395836 26115 raft_consensus.cc:2781] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:04:45.395900 25915 raft_consensus.cc:2454] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb516e6024c34e32a9b6fc0c7370d0f0 in term 0.
I20260929 16:04:45.395987 26115 raft_consensus.cc:482] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:04:45.396209 26115 raft_consensus.cc:3037] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:45.398746 26115 raft_consensus.cc:504] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 } } peers { permanent_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45775 } } peers { permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 } }
I20260929 16:04:45.399433 26115 leader_election.cc:288] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [CANDIDATE]: Term 1 election: Requested vote from peers 8491613c3ca74a5db113badfb73bbaf8 (127.20.121.129:37415), 8d79dcb893ec46cb82dd16907a97e0e7 (127.20.121.130:43697)
I20260929 16:04:45.400184 25784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7088b29302694d67b6bf56d4801d8045" candidate_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8491613c3ca74a5db113badfb73bbaf8"
I20260929 16:04:45.400220 25915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7088b29302694d67b6bf56d4801d8045" candidate_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d79dcb893ec46cb82dd16907a97e0e7"
I20260929 16:04:45.400449 25915 raft_consensus.cc:3037] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:45.400441 25784 raft_consensus.cc:3037] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:45.403981 25784 raft_consensus.cc:2454] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb516e6024c34e32a9b6fc0c7370d0f0 in term 1.
I20260929 16:04:45.403981 25915 raft_consensus.cc:2454] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb516e6024c34e32a9b6fc0c7370d0f0 in term 1.
I20260929 16:04:45.404606 25981 leader_election.cc:302] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d79dcb893ec46cb82dd16907a97e0e7, fb516e6024c34e32a9b6fc0c7370d0f0; no voters:
I20260929 16:04:45.405028 26115 raft_consensus.cc:2781] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:45.405603 26115 raft_consensus.cc:686] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 1 LEADER]: Becoming Leader. State: Replica: fb516e6024c34e32a9b6fc0c7370d0f0, State: Running, Role: LEADER
I20260929 16:04:45.406322 26115 consensus_queue.cc:237] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 } } peers { permanent_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45775 } } peers { permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 } }
I20260929 16:04:45.411607 25643 catalog_manager.cc:5495] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb516e6024c34e32a9b6fc0c7370d0f0 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb516e6024c34e32a9b6fc0c7370d0f0" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 } health_report { overall_health: UNKNOWN } } }
I20260929 16:04:45.435708 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.432928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.20.121.131:45775)
0929 16:04:45.433138 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.433143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.433163 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:04:45.433365 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.433369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.433455 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.433550 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.434759 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.434764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.434767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.434785 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.434819 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.434833 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.434980 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.435225 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.435416 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260929 16:04:45.445833 25784 raft_consensus.cc:1261] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8 [term 1 FOLLOWER]: Refusing update from remote peer fb516e6024c34e32a9b6fc0c7370d0f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:45.445830 25915 raft_consensus.cc:1261] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7 [term 1 FOLLOWER]: Refusing update from remote peer fb516e6024c34e32a9b6fc0c7370d0f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:04:45.447101 26118 consensus_queue.cc:1035] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8491613c3ca74a5db113badfb73bbaf8" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 37415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:45.447602 26115 consensus_queue.cc:1035] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d79dcb893ec46cb82dd16907a97e0e7" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 43697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:45.460672 26121 mvcc.cc:205] Tried to move back new op lower bound from 7334698538773622784 to 7334698538629087232. Current Snapshot: MvccSnapshot[applied={T|T < 7334698538773622784}]
I20260929 16:04:45.479283 25642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:32800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:45.479637 25642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:32800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:04:45.483980 25642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b06f8aad25042cca99ec7c764efedf8: Sending DeleteTablet for 3 replicas of tablet 7088b29302694d67b6bf56d4801d8045
I20260929 16:04:45.485657 25764 tablet_service.cc:1514] Processing DeleteTablet for tablet 7088b29302694d67b6bf56d4801d8045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:45 UTC) from {username='slave'} at 127.0.0.1:45460
I20260929 16:04:45.485742 26025 tablet_service.cc:1514] Processing DeleteTablet for tablet 7088b29302694d67b6bf56d4801d8045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:45 UTC) from {username='slave'} at 127.0.0.1:36362
I20260929 16:04:45.486159 25895 tablet_service.cc:1514] Processing DeleteTablet for tablet 7088b29302694d67b6bf56d4801d8045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:04:45 UTC) from {username='slave'} at 127.0.0.1:37446
I20260929 16:04:45.488265 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 25702
I20260929 16:04:45.501386 26133 tablet_replica.cc:330] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: stopping tablet replica
I20260929 16:04:45.502118 26133 raft_consensus.cc:2229] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:45.502820 26133 raft_consensus.cc:2258] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:45.504592 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.490673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.20.121.129:37415)
0929 16:04:45.490920 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.490930 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.501633 (+ 10703us) server_negotiation.cc:407] Connection header received
0929 16:04:45.501952 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.501959 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.502133 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.502220 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.503537 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.503543 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.503549 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.503571 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.503616 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.503633 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.503754 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.504140 (+ 386us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.504237 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 16:04:45.505998 25824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:45.506979 26131 tablet_replica.cc:330] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: stopping tablet replica
I20260929 16:04:45.507547 26131 raft_consensus.cc:2229] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:04:45.508584 26131 raft_consensus.cc:2258] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:45.509280 26133 ts_tablet_manager.cc:1905] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:45.511240 26131 ts_tablet_manager.cc:1905] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:45.519448 26131 ts_tablet_manager.cc:1918] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:04:45.519706 26131 log.cc:1196] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/7088b29302694d67b6bf56d4801d8045
I20260929 16:04:45.520329 26131 ts_tablet_manager.cc:1939] T 7088b29302694d67b6bf56d4801d8045 P fb516e6024c34e32a9b6fc0c7370d0f0: Deleting consensus metadata
I20260929 16:04:45.520359 26133 ts_tablet_manager.cc:1918] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:04:45.520751 26133 log.cc:1196] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/7088b29302694d67b6bf56d4801d8045
I20260929 16:04:45.521441 26133 ts_tablet_manager.cc:1939] T 7088b29302694d67b6bf56d4801d8045 P 8d79dcb893ec46cb82dd16907a97e0e7: Deleting consensus metadata
I20260929 16:04:45.522421 25630 catalog_manager.cc:4841] TS fb516e6024c34e32a9b6fc0c7370d0f0 (127.20.121.131:45775): tablet 7088b29302694d67b6bf56d4801d8045 (table test-table [id=7df152bc0ee24cadb149240035fa6bb4]) successfully deleted
I20260929 16:04:45.523695 25630 catalog_manager.cc:4841] TS 8d79dcb893ec46cb82dd16907a97e0e7 (127.20.121.130:43697): tablet 7088b29302694d67b6bf56d4801d8045 (table test-table [id=7df152bc0ee24cadb149240035fa6bb4]) successfully deleted
I20260929 16:04:45.637521 26132 tablet_replica.cc:330] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8: stopping tablet replica
I20260929 16:04:45.639039 26132 raft_consensus.cc:2229] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:45.639561 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 25702
I20260929 16:04:45.639654 26132 raft_consensus.cc:2258] T 7088b29302694d67b6bf56d4801d8045 P 8491613c3ca74a5db113badfb73bbaf8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 16:04:45.663010 25627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.121.129:37415 (error 108)
I20260929 16:04:45.663949 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 25833
I20260929 16:04:45.666464 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.665792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:37415 (local address 127.0.0.1:45476)
0929 16:04:45.666111 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.666262 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
W20260929 16:04:45.666761 25627 catalog_manager.cc:4584] TS 8491613c3ca74a5db113badfb73bbaf8 (127.20.121.129:37415): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7088b29302694d67b6bf56d4801d8045: Network error: Client connection negotiation failed: client connection to 127.20.121.129:37415: connect: Connection refused (error 111)
I20260929 16:04:45.667876 26106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.665239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37450 (local address 127.20.121.130:43697)
0929 16:04:45.665408 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.665414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.665436 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:04:45.665626 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.665632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.665739 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.665855 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.667173 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.667178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.667181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.667199 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.667232 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.667244 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.667397 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.667524 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.667589 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 16:04:45.668546 25955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:45.720690 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.720090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:37415 (local address 127.0.0.1:45488)
0929 16:04:45.720385 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.720500 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260929 16:04:45.722071 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.721559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:37415 (local address 127.0.0.1:45504)
0929 16:04:45.721737 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.721846 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 16:04:45.775993 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 25833
I20260929 16:04:45.783143 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.782522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:37415 (local address 127.0.0.1:45508)
0929 16:04:45.782780 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.782918 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260929 16:04:45.784456 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.783976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:37415 (local address 127.0.0.1:45514)
0929 16:04:45.784172 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.784270 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 16:04:45.794051 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 25963
I20260929 16:04:45.797415 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.794930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.20.121.131:45775)
0929 16:04:45.795142 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:45.795148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:45.795171 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:04:45.795277 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:45.795283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:45.795347 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:45.795467 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:45.796561 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:45.796574 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:45.796580 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:45.796607 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:45.796650 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:45.796668 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:45.796834 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:45.796975 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 16:04:45.797075 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 16:04:45.798382 26085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:45.865412 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.864934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:37415 (local address 127.0.0.1:45524)
0929 16:04:45.865171 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.865253 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260929 16:04:45.866745 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:45.866290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:37415 (local address 127.0.0.1:45528)
0929 16:04:45.866531 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:45.866614 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260929 16:04:45.920603 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 25963
I20260929 16:04:45.941838 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 25612
I20260929 16:04:45.943135 25673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:46.051430 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:46.050869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:37415 (local address 127.0.0.1:45530)
0929 16:04:46.051085 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:46.051200 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:37415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 16:04:46.051993 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 25612
2026-09-29T16:04:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2904 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 16:04:46.095932 20966 mini_oidc.cc:152] Starting JWKS server
I20260929 16:04:46.098831 20966 webserver.cc:460] Webserver started at https://127.0.0.1:38477/ using document root <none> and password file <none>
I20260929 16:04:46.099166 20966 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 16:04:46.099901 20966 webserver.cc:460] Webserver started at http://127.0.0.1:36903/ using document root <none> and password file <none>
2026-09-29T16:04:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:46Z Disabled control of system clock
I20260929 16:04:46.129317 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37983
--webserver_interface=127.20.121.190
--webserver_port=0
--builtin_ntp_servers=127.20.121.148:43785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:37983
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38477/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/testchainca.cert with env {}
W20260929 16:04:46.483947 26145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:46.484285 26145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:46.484377 26145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:46.492220 26145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:46.492357 26145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:46.492405 26145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:46.492455 26145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:46.492493 26145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 16:04:46.492547 26145 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 16:04:46.492594 26145 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38477/jwks.json
I20260929 16:04:46.503574 26145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:37983
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38477/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:46.504884 26145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:46.506271 26145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:46.518577 26151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:46.518929 26150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:46.518958 26153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:46.520442 26145 server_base.cc:1034] running on GCE node
I20260929 16:04:46.521387 26145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:46.523048 26145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:46.524298 26145 hybrid_clock.cc:648] HybridClock initialized: now 1790697886524233 us; error 55 us; skew 500 ppm
I20260929 16:04:46.524667 26145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:04:46.527267 26145 webserver.cc:460] Webserver started at http://127.20.121.190:35269/ using document root <none> and password file <none>
I20260929 16:04:46.527959 26145 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:46.528051 26145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:46.528342 26145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:46.531297 26145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "3165ede566d24792a572bcc9b9d9fb01"
format_stamp: "Formatted at 2026-09-29 16:04:46 on dist-test-slave-x07d"
server_key: "8f95df635301a3d09c4c8768adf32a6c"
server_key_iv: "d54c292b7798f626657afb0446f41bfe"
server_key_version: "encryptionkey@0"
I20260929 16:04:46.532034 26145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "3165ede566d24792a572bcc9b9d9fb01"
format_stamp: "Formatted at 2026-09-29 16:04:46 on dist-test-slave-x07d"
server_key: "8f95df635301a3d09c4c8768adf32a6c"
server_key_iv: "d54c292b7798f626657afb0446f41bfe"
server_key_version: "encryptionkey@0"
I20260929 16:04:46.537137 26145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260929 16:04:46.540693 26159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:46.542194 26145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:46.542405 26145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "3165ede566d24792a572bcc9b9d9fb01"
format_stamp: "Formatted at 2026-09-29 16:04:46 on dist-test-slave-x07d"
server_key: "8f95df635301a3d09c4c8768adf32a6c"
server_key_iv: "d54c292b7798f626657afb0446f41bfe"
server_key_version: "encryptionkey@0"
I20260929 16:04:46.542629 26145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:46.559059 26145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:46.559871 26145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:46.560125 26145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:46.578986 26145 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:37983
I20260929 16:04:46.579171 26210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:37983 every 8 connection(s)
I20260929 16:04:46.580387 26145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:46.584436 26211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:46.590574 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 26145
I20260929 16:04:46.590801 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:46.591535 20966 external_mini_cluster.cc:1428] Setting key a5bff549792b89fab666ad4287d90046
I20260929 16:04:46.596920 26211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01: Bootstrap starting.
I20260929 16:04:46.600500 26211 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:46.601783 26211 log.cc:824] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:46.605443 26211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01: No bootstrap required, opened a new log
I20260929 16:04:46.607218 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:46.593765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56884 (local address 127.20.121.190:37983)
0929 16:04:46.594955 (+ 1190us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:46.594968 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:46.595015 (+ 47us) server_negotiation.cc:407] Connection header received
0929 16:04:46.596945 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:46.596971 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:46.597606 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:46.598120 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:46.598782 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:46.600274 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:46.601992 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:46.603180 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:46.603391 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:46.604785 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:46.604793 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:46.604808 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:46.605390 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:46.605480 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:46.605501 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:46.605682 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:46.605938 (+ 256us) server_negotiation.cc:299] Negotiation successful
0929 16:04:46.606272 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":449,"threads_started":1}
I20260929 16:04:46.611228 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:04:46.613746 26211 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165ede566d24792a572bcc9b9d9fb01" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37983 } }
I20260929 16:04:46.614164 26211 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:46.614293 26211 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3165ede566d24792a572bcc9b9d9fb01, State: Initialized, Role: FOLLOWER
I20260929 16:04:46.614930 26211 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165ede566d24792a572bcc9b9d9fb01" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37983 } }
I20260929 16:04:46.615134 26211 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:46.615276 26211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:46.615448 26211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:46.617563 26211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165ede566d24792a572bcc9b9d9fb01" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37983 } }
I20260929 16:04:46.618117 26211 leader_election.cc:302] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3165ede566d24792a572bcc9b9d9fb01; no voters:
I20260929 16:04:46.618691 26211 leader_election.cc:288] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:46.618860 26216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:46.619613 26216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [term 1 LEADER]: Becoming Leader. State: Replica: 3165ede566d24792a572bcc9b9d9fb01, State: Running, Role: LEADER
I20260929 16:04:46.620167 26216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165ede566d24792a572bcc9b9d9fb01" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37983 } }
I20260929 16:04:46.621027 26211 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:46.623343 26217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3165ede566d24792a572bcc9b9d9fb01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165ede566d24792a572bcc9b9d9fb01" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37983 } } }
I20260929 16:04:46.623323 26218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3165ede566d24792a572bcc9b9d9fb01. Latest consensus state: current_term: 1 leader_uuid: "3165ede566d24792a572bcc9b9d9fb01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3165ede566d24792a572bcc9b9d9fb01" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37983 } } }
I20260929 16:04:46.623792 26217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:46.623813 26218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:46.627385 26221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:46.633270 26221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:46.634073 26233 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 16:04:46.642750 26221 catalog_manager.cc:1347] Generated new cluster ID: d3f298a3dbbc468b9a8a08a5460b298a
I20260929 16:04:46.642910 26221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:46.654670 26221 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:46.656033 26221 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:46.665877 26221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3165ede566d24792a572bcc9b9d9fb01: Generated new TSK 0
I20260929 16:04:46.666913 26221 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 16:04:46.704032 26241 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 16:04:46.704229 26241 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 16:04:46.705144 26214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:46.683630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56886 (local address 127.20.121.190:37983)
0929 16:04:46.683878 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:46.683886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:46.684329 (+ 443us) server_negotiation.cc:407] Connection header received
0929 16:04:46.684675 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:46.684678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:46.684745 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:46.684851 (+ 106us) server_negotiation.cc:226] Negotiated authn=JWT
0929 16:04:46.685483 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:46.686308 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:46.688421 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:46.688890 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:46.689000 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:46.689616 (+ 616us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 16:04:46.704638 (+ 15022us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38477/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 16:04:46.704935 (+ 297us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56886: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38477/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260929 16:04:46.705307 26214 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56886: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38477/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 16:04:46.707193 26240 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:04:46.683498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37983 (local address 127.0.0.1:56886)
0929 16:04:46.684213 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:46.684235 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:46.684430 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:46.684910 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:46.684915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:46.684929 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0929 16:04:46.685330 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:46.685344 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:46.686446 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:46.686452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:46.688257 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:46.688272 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:46.689089 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:46.689097 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:46.689283 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:46.689367 (+ 84us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 16:04:46.704964 (+ 15597us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38477/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 16:04:46.705356 (+ 392us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.121.190:37983: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38477/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":365,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.20.121.190:37983: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38477/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 16:04:46.709033 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 26145
I20260929 16:04:46.710418 26206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:46.802121 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 26145
2026-09-29T16:04:46Z chronyd exiting
I20260929 16:04:46.852936 20966 test_util.cc:175] -----------------------------------------------
I20260929 16:04:46.853068 20966 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790697841464038-20966-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (761 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 16:04:46.856895 20966 mini_oidc.cc:152] Starting JWKS server
I20260929 16:04:46.858778 20966 webserver.cc:460] Webserver started at https://127.0.0.1:33571/ using document root <none> and password file <none>
I20260929 16:04:46.859033 20966 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 16:04:46.859756 20966 webserver.cc:460] Webserver started at http://127.0.0.1:35815/ using document root <none> and password file <none>
2026-09-29T16:04:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:46Z Disabled control of system clock
I20260929 16:04:46.889338 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:35145
--webserver_interface=127.20.121.190
--webserver_port=0
--builtin_ntp_servers=127.20.121.148:34009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:35145
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33571/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/testchainca.cert with env {}
W20260929 16:04:47.219825 26248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:47.220137 26248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:47.220227 26248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:47.227641 26248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:47.227763 26248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:47.227801 26248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:47.227851 26248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:47.227895 26248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 16:04:47.227957 26248 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 16:04:47.227998 26248 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33571/jwks.json
I20260929 16:04:47.238308 26248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:34009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:35145
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:35145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33571/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:47.239769 26248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:47.241086 26248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:47.253726 26254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:47.253693 26253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:47.253779 26256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:47.254597 26248 server_base.cc:1034] running on GCE node
I20260929 16:04:47.255426 26248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:47.256916 26248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:47.258140 26248 hybrid_clock.cc:648] HybridClock initialized: now 1790697887258104 us; error 30 us; skew 500 ppm
I20260929 16:04:47.258462 26248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:04:47.261267 26248 webserver.cc:460] Webserver started at http://127.20.121.190:38899/ using document root <none> and password file <none>
I20260929 16:04:47.261986 26248 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:47.262182 26248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:47.262470 26248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:47.265547 26248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "45cf0efdd79c48aa902e25960c6791f3"
format_stamp: "Formatted at 2026-09-29 16:04:47 on dist-test-slave-x07d"
server_key: "c42883c7e3d116d5b2a6889b123007fd"
server_key_iv: "34bccb04bbba2430b398731b39283a49"
server_key_version: "encryptionkey@0"
I20260929 16:04:47.266419 26248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "45cf0efdd79c48aa902e25960c6791f3"
format_stamp: "Formatted at 2026-09-29 16:04:47 on dist-test-slave-x07d"
server_key: "c42883c7e3d116d5b2a6889b123007fd"
server_key_iv: "34bccb04bbba2430b398731b39283a49"
server_key_version: "encryptionkey@0"
I20260929 16:04:47.271906 26248 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 16:04:47.275502 26262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:47.276962 26248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260929 16:04:47.277173 26248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "45cf0efdd79c48aa902e25960c6791f3"
format_stamp: "Formatted at 2026-09-29 16:04:47 on dist-test-slave-x07d"
server_key: "c42883c7e3d116d5b2a6889b123007fd"
server_key_iv: "34bccb04bbba2430b398731b39283a49"
server_key_version: "encryptionkey@0"
I20260929 16:04:47.277392 26248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:47.292424 26248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:47.293286 26248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:47.293599 26248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:47.312505 26248 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:35145
I20260929 16:04:47.312526 26313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:35145 every 8 connection(s)
I20260929 16:04:47.313912 26248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:47.317649 26314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:47.320950 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 26248
I20260929 16:04:47.321174 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:47.321825 20966 external_mini_cluster.cc:1428] Setting key ee02a9edc9fb3cff988ca2b1381a2dd7
I20260929 16:04:47.329875 26314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3: Bootstrap starting.
I20260929 16:04:47.333079 26314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:47.334344 26314 log.cc:824] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:47.335175 26317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:47.323702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40922 (local address 127.20.121.190:35145)
0929 16:04:47.324646 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:47.324657 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:47.324698 (+ 41us) server_negotiation.cc:407] Connection header received
0929 16:04:47.326379 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:47.326408 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:47.326896 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:47.327315 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:47.327887 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:47.329171 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:47.330528 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:47.331217 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:47.331473 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:47.332641 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:47.332649 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:47.332663 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:47.333209 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:47.333368 (+ 159us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:47.333392 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:47.333529 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:47.333767 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 16:04:47.334151 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":377,"threads_started":1}
I20260929 16:04:47.337345 26314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3: No bootstrap required, opened a new log
I20260929 16:04:47.339052 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:04:47.346613 26314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0efdd79c48aa902e25960c6791f3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35145 } }
I20260929 16:04:47.347028 26314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:47.347152 26314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45cf0efdd79c48aa902e25960c6791f3, State: Initialized, Role: FOLLOWER
I20260929 16:04:47.347857 26314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0efdd79c48aa902e25960c6791f3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35145 } }
I20260929 16:04:47.348155 26314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:47.348308 26314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:47.348483 26314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:47.350806 26314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0efdd79c48aa902e25960c6791f3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35145 } }
I20260929 16:04:47.351344 26314 leader_election.cc:302] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45cf0efdd79c48aa902e25960c6791f3; no voters:
I20260929 16:04:47.352001 26314 leader_election.cc:288] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:47.352140 26319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:47.352911 26319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [term 1 LEADER]: Becoming Leader. State: Replica: 45cf0efdd79c48aa902e25960c6791f3, State: Running, Role: LEADER
I20260929 16:04:47.353430 26319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0efdd79c48aa902e25960c6791f3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35145 } }
I20260929 16:04:47.354324 26314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:47.356506 26321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45cf0efdd79c48aa902e25960c6791f3. Latest consensus state: current_term: 1 leader_uuid: "45cf0efdd79c48aa902e25960c6791f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0efdd79c48aa902e25960c6791f3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35145 } } }
I20260929 16:04:47.356981 26321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:47.359179 26325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:47.359193 26320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45cf0efdd79c48aa902e25960c6791f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45cf0efdd79c48aa902e25960c6791f3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35145 } } }
I20260929 16:04:47.359642 26320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3 [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:47.364406 26325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:47.373832 26325 catalog_manager.cc:1347] Generated new cluster ID: 354eb83934a14b7999f41d13f3e493db
I20260929 16:04:47.373958 26325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:47.397466 26325 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:47.399361 26325 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:47.415007 26325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45cf0efdd79c48aa902e25960c6791f3: Generated new TSK 0
I20260929 16:04:47.415917 26325 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 16:04:47.452905 26344 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 16:04:47.453054 26344 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 16:04:47.454479 26317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:47.433601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.20.121.190:35145)
0929 16:04:47.433800 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:47.433807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:47.434203 (+ 396us) server_negotiation.cc:407] Connection header received
0929 16:04:47.434439 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:47.434446 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:47.434550 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:47.434737 (+ 187us) server_negotiation.cc:226] Negotiated authn=JWT
0929 16:04:47.435161 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:47.436315 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:47.437696 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:47.438428 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:47.438560 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:47.439042 (+ 482us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 16:04:47.453656 (+ 14614us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33571/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 16:04:47.454169 (+ 513us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40924: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33571/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":70}
W20260929 16:04:47.454627 26317 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40924: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33571/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260929 16:04:47.454766 26343 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:04:47.433449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:35145 (local address 127.0.0.1:40924)
0929 16:04:47.434083 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:47.434105 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:47.434246 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:47.434749 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:47.434752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:47.434761 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0929 16:04:47.435005 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:47.435012 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:47.436479 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:47.436483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:47.437550 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:47.437562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:47.438605 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:47.438610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:47.438741 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:47.438774 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 16:04:47.453927 (+ 15153us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33571/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 16:04:47.454281 (+ 354us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.121.190:35145: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33571/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":323,"threads_started":1}
I20260929 16:04:47.456471 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 26248
I20260929 16:04:47.457659 26309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:47.552503 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 26248
2026-09-29T16:04:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (751 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 16:04:47.608877 20966 mini_oidc.cc:152] Starting JWKS server
I20260929 16:04:47.611001 20966 webserver.cc:460] Webserver started at https://127.0.0.1:38697/ using document root <none> and password file <none>
I20260929 16:04:47.611266 20966 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 16:04:47.612061 20966 webserver.cc:460] Webserver started at http://127.0.0.1:38887/ using document root <none> and password file <none>
2026-09-29T16:04:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:47Z Disabled control of system clock
I20260929 16:04:47.641973 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:42597
--webserver_interface=127.20.121.190
--webserver_port=0
--builtin_ntp_servers=127.20.121.148:39219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:42597
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38697/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 16:04:47.975612 26351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:47.975997 26351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:47.976162 26351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:47.983629 26351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:47.983795 26351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:47.983887 26351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:47.983997 26351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:47.984079 26351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 16:04:47.984174 26351 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 16:04:47.984229 26351 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38697/jwks.json
I20260929 16:04:47.993734 26351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:39219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:42597
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:42597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38697/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:47.995136 26351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:47.996937 26351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:48.009682 26356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:48.009708 26359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:48.009817 26357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:48.010538 26351 server_base.cc:1034] running on GCE node
I20260929 16:04:48.011422 26351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:48.012992 26351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:48.014287 26351 hybrid_clock.cc:648] HybridClock initialized: now 1790697888014220 us; error 62 us; skew 500 ppm
I20260929 16:04:48.014724 26351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:04:48.017450 26351 webserver.cc:460] Webserver started at http://127.20.121.190:33377/ using document root <none> and password file <none>
I20260929 16:04:48.018411 26351 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:48.018564 26351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:48.018918 26351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:48.022300 26351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "8903970454f94b08b74c64a71c211d3a"
format_stamp: "Formatted at 2026-09-29 16:04:48 on dist-test-slave-x07d"
server_key: "e243918135f091ee7fe4b2a6b1fd898f"
server_key_iv: "7e4b55b8c45e6c9d0e196ccd24eb1c02"
server_key_version: "encryptionkey@0"
I20260929 16:04:48.023249 26351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "8903970454f94b08b74c64a71c211d3a"
format_stamp: "Formatted at 2026-09-29 16:04:48 on dist-test-slave-x07d"
server_key: "e243918135f091ee7fe4b2a6b1fd898f"
server_key_iv: "7e4b55b8c45e6c9d0e196ccd24eb1c02"
server_key_version: "encryptionkey@0"
I20260929 16:04:48.029196 26351 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260929 16:04:48.033166 26365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:48.034979 26351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:48.035305 26351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "8903970454f94b08b74c64a71c211d3a"
format_stamp: "Formatted at 2026-09-29 16:04:48 on dist-test-slave-x07d"
server_key: "e243918135f091ee7fe4b2a6b1fd898f"
server_key_iv: "7e4b55b8c45e6c9d0e196ccd24eb1c02"
server_key_version: "encryptionkey@0"
I20260929 16:04:48.035567 26351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:48.066272 26351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:48.067167 26351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:48.067502 26351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:48.087164 26351 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:42597
I20260929 16:04:48.087199 26416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:42597 every 8 connection(s)
I20260929 16:04:48.088609 26351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:48.092748 26417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:48.094250 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 26351
I20260929 16:04:48.094465 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:48.095278 20966 external_mini_cluster.cc:1428] Setting key c869bbab1fdabbc455ce988c9bd7a3a5
I20260929 16:04:48.105136 26417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a: Bootstrap starting.
I20260929 16:04:48.108444 26417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:48.109432 26420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:48.097509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.20.121.190:42597)
0929 16:04:48.098664 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:48.098685 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:48.098728 (+ 43us) server_negotiation.cc:407] Connection header received
0929 16:04:48.100422 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:48.100448 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:48.100946 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:48.101344 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:48.101923 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:48.103374 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:48.104764 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:48.105612 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:48.105812 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:48.107031 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:48.107044 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:48.107059 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:48.107604 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:48.107700 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:48.107721 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:48.107850 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:48.108087 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 16:04:48.108395 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":465,"threads_started":1}
I20260929 16:04:48.109524 26417 log.cc:824] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:48.113157 26417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a: No bootstrap required, opened a new log
I20260929 16:04:48.113992 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:04:48.121281 26417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8903970454f94b08b74c64a71c211d3a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42597 } }
I20260929 16:04:48.121752 26417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:48.121896 26417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8903970454f94b08b74c64a71c211d3a, State: Initialized, Role: FOLLOWER
I20260929 16:04:48.122496 26417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8903970454f94b08b74c64a71c211d3a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42597 } }
I20260929 16:04:48.122732 26417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:48.122875 26417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:48.123049 26417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:48.125206 26417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8903970454f94b08b74c64a71c211d3a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42597 } }
I20260929 16:04:48.125738 26417 leader_election.cc:302] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8903970454f94b08b74c64a71c211d3a; no voters:
I20260929 16:04:48.126750 26417 leader_election.cc:288] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:48.126998 26422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:48.127751 26422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [term 1 LEADER]: Becoming Leader. State: Replica: 8903970454f94b08b74c64a71c211d3a, State: Running, Role: LEADER
I20260929 16:04:48.128315 26422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8903970454f94b08b74c64a71c211d3a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42597 } }
I20260929 16:04:48.129099 26417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:48.131393 26424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8903970454f94b08b74c64a71c211d3a. Latest consensus state: current_term: 1 leader_uuid: "8903970454f94b08b74c64a71c211d3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8903970454f94b08b74c64a71c211d3a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42597 } } }
I20260929 16:04:48.132026 26424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:48.131538 26423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8903970454f94b08b74c64a71c211d3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8903970454f94b08b74c64a71c211d3a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42597 } } }
I20260929 16:04:48.132302 26423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:48.136061 26431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:48.141027 26431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:48.151165 26431 catalog_manager.cc:1347] Generated new cluster ID: fddb79fa998a44c197e22e48fa42618b
I20260929 16:04:48.151297 26431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:48.164678 26431 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:48.166146 26431 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:48.179584 26431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8903970454f94b08b74c64a71c211d3a: Generated new TSK 0
I20260929 16:04:48.180559 26431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:04:48.212132 26420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:48.204622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48550 (local address 127.20.121.190:42597)
0929 16:04:48.204843 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:48.204851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:48.205184 (+ 333us) server_negotiation.cc:407] Connection header received
0929 16:04:48.205460 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:48.205467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:48.205550 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:48.205716 (+ 166us) server_negotiation.cc:226] Negotiated authn=JWT
0929 16:04:48.206286 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:48.206998 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:48.208378 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:48.208838 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:48.208976 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:48.209494 (+ 518us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 16:04:48.211458 (+ 1964us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38697/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 16:04:48.211798 (+ 340us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48550: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38697/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":59}
W20260929 16:04:48.212263 26446 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:04:48.204517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:42597 (local address 127.0.0.1:48550)
0929 16:04:48.205070 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:48.205090 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:48.205262 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:48.205726 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:48.205730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:48.205742 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0929 16:04:48.206134 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:48.206143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:48.207166 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:48.207173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:48.208234 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:48.208245 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:48.209056 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:48.209063 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:48.209206 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:48.209240 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 16:04:48.211651 (+ 2411us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38697/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 16:04:48.211883 (+ 232us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.121.190:42597: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38697/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":308,"threads_started":1}
W20260929 16:04:48.212276 26420 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48550: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38697/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260929 16:04:48.213316 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 26351
I20260929 16:04:48.214629 26412 generic_service.cc:196] Checking for leaks (request via RPC)
E20260929 16:04:48.231062 26447 webserver.cc:533] Webserver: sslize failed:
E20260929 16:04:48.231200 26447 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 16:04:48.311780 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 26351
2026-09-29T16:04:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (758 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T16:04:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:48Z Disabled control of system clock
I20260929 16:04:48.395462 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:46003
--webserver_interface=127.20.121.190
--webserver_port=0
--builtin_ntp_servers=127.20.121.148:36371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:46003
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/testchainca.cert with env {}
W20260929 16:04:48.756333 26452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:48.756659 26452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:48.756801 26452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:48.764595 26452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:48.764752 26452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:48.764842 26452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:48.764966 26452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:48.765058 26452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 16:04:48.765164 26452 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 16:04:48.765223 26452 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 16:04:48.776271 26452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:36371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:46003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:46003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:48.777707 26452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:48.779126 26452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:48.792240 26458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:48.792411 26457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:48.792464 26452 server_base.cc:1034] running on GCE node
W20260929 16:04:48.792415 26460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:48.793608 26452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:48.795251 26452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:48.796500 26452 hybrid_clock.cc:648] HybridClock initialized: now 1790697888796468 us; error 38 us; skew 500 ppm
I20260929 16:04:48.796913 26452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:04:48.799542 26452 webserver.cc:460] Webserver started at http://127.20.121.190:40833/ using document root <none> and password file <none>
I20260929 16:04:48.800343 26452 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:48.800488 26452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:48.800809 26452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:48.803925 26452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "72b8e7c259fb45c09723a6db29251a5c"
format_stamp: "Formatted at 2026-09-29 16:04:48 on dist-test-slave-x07d"
server_key: "ce1f92b222c98db7481cf60df1ed3e6a"
server_key_iv: "ff7d89b275ced60c0cec6cca6d344db4"
server_key_version: "encryptionkey@0"
I20260929 16:04:48.804822 26452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "72b8e7c259fb45c09723a6db29251a5c"
format_stamp: "Formatted at 2026-09-29 16:04:48 on dist-test-slave-x07d"
server_key: "ce1f92b222c98db7481cf60df1ed3e6a"
server_key_iv: "ff7d89b275ced60c0cec6cca6d344db4"
server_key_version: "encryptionkey@0"
I20260929 16:04:48.810600 26452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 16:04:48.814365 26466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:48.816190 26452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260929 16:04:48.816457 26452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "72b8e7c259fb45c09723a6db29251a5c"
format_stamp: "Formatted at 2026-09-29 16:04:48 on dist-test-slave-x07d"
server_key: "ce1f92b222c98db7481cf60df1ed3e6a"
server_key_iv: "ff7d89b275ced60c0cec6cca6d344db4"
server_key_version: "encryptionkey@0"
I20260929 16:04:48.816736 26452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:48.845269 26452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:48.846268 26452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:48.846616 26452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:48.867429 26452 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:46003
I20260929 16:04:48.867450 26517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:46003 every 8 connection(s)
I20260929 16:04:48.868892 26452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:48.872790 26518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:48.878295 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 26452
I20260929 16:04:48.878546 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:48.879226 20966 external_mini_cluster.cc:1428] Setting key e435b89808e3a79d6236dc27dbc71440
I20260929 16:04:48.885540 26518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c: Bootstrap starting.
I20260929 16:04:48.888950 26518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:48.890067 26518 log.cc:824] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:48.893352 26518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c: No bootstrap required, opened a new log
I20260929 16:04:48.895525 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:48.882218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57254 (local address 127.20.121.190:46003)
0929 16:04:48.883614 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:48.883630 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:48.883720 (+ 90us) server_negotiation.cc:407] Connection header received
0929 16:04:48.886329 (+ 2609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:48.886353 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:48.886834 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:48.887270 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:48.887832 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:48.889185 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:48.890638 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:48.891502 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:48.891723 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:48.892978 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:48.892986 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:48.893000 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:04:48.893604 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:48.893683 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:04:48.893704 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:48.893814 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:48.894169 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 16:04:48.894510 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":519,"threads_started":1}
I20260929 16:04:48.900044 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:04:48.903657 26518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b8e7c259fb45c09723a6db29251a5c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 46003 } }
I20260929 16:04:48.904096 26518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:48.904232 26518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72b8e7c259fb45c09723a6db29251a5c, State: Initialized, Role: FOLLOWER
I20260929 16:04:48.904803 26518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b8e7c259fb45c09723a6db29251a5c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 46003 } }
I20260929 16:04:48.905048 26518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:48.905193 26518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:48.905375 26518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:48.907827 26518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b8e7c259fb45c09723a6db29251a5c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 46003 } }
I20260929 16:04:48.908355 26518 leader_election.cc:302] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72b8e7c259fb45c09723a6db29251a5c; no voters:
I20260929 16:04:48.908958 26518 leader_election.cc:288] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:48.909159 26523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:48.910332 26523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [term 1 LEADER]: Becoming Leader. State: Replica: 72b8e7c259fb45c09723a6db29251a5c, State: Running, Role: LEADER
I20260929 16:04:48.910940 26523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b8e7c259fb45c09723a6db29251a5c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 46003 } }
I20260929 16:04:48.911288 26518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:48.914111 26525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72b8e7c259fb45c09723a6db29251a5c. Latest consensus state: current_term: 1 leader_uuid: "72b8e7c259fb45c09723a6db29251a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b8e7c259fb45c09723a6db29251a5c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 46003 } } }
I20260929 16:04:48.914508 26525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:48.914081 26524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72b8e7c259fb45c09723a6db29251a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b8e7c259fb45c09723a6db29251a5c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 46003 } } }
I20260929 16:04:48.915153 26524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:48.919255 26530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:48.925000 26530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:48.935164 26530 catalog_manager.cc:1347] Generated new cluster ID: 00022614cc3c4d6da397cdcc52fc2039
I20260929 16:04:48.935304 26530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:48.964309 26530 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:48.965698 26530 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:48.980962 26530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72b8e7c259fb45c09723a6db29251a5c: Generated new TSK 0
I20260929 16:04:48.982172 26530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:04:49.087495 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:49.007118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57260 (local address 127.20.121.190:46003)
0929 16:04:49.007372 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:49.007405 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:49.007736 (+ 331us) server_negotiation.cc:407] Connection header received
0929 16:04:49.008048 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:49.008053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:49.008129 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:49.008312 (+ 183us) server_negotiation.cc:226] Negotiated authn=JWT
0929 16:04:49.008775 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:49.009545 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:49.011304 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:49.011886 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:49.012040 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:49.012562 (+ 522us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 16:04:49.086621 (+ 74059us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 16:04:49.087115 (+ 494us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57260: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 16:04:49.087657 26521 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57260: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 16:04:49.087729 26547 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:04:49.006860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:46003 (local address 127.0.0.1:57260)
0929 16:04:49.007607 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:49.007623 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:49.007814 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:49.008322 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:49.008326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:49.008338 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0929 16:04:49.008613 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:49.008621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:49.009878 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:49.009884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:49.011135 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:49.011150 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:49.012082 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:49.012089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:49.012249 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:49.012285 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 16:04:49.086949 (+ 74664us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 16:04:49.087293 (+ 344us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.121.190:46003: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":355,"threads_started":1}
I20260929 16:04:49.089386 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 26452
I20260929 16:04:49.090811 26513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:49.196597 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 26452
2026-09-29T16:04:49Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (866 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (416 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (438 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:50 dist-test-slave-x07d krb5kdc[26562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:50 dist-test-slave-x07d krb5kdc[26562](info): set up 2 sockets
Sep 29 16:04:50 dist-test-slave-x07d krb5kdc[26562](info): commencing operation
krb5kdc: starting...
W20260929 16:04:51.211182 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:51 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:04:51.438423 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34291
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:34291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:51.775305 26575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:51.775637 26575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:51.775790 26575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:51.783578 26575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:51.783718 26575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:51.783808 26575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:51.783907 26575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:51.783951 26575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:51.794361 26575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:34291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:51.795796 26575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:51.797211 26575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:51.810899 26580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:51.811002 26581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:51.811033 26583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:51.816797 26575 server_base.cc:1034] running on GCE node
I20260929 16:04:51.817922 26575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:51.819737 26575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:51.821025 26575 hybrid_clock.cc:648] HybridClock initialized: now 1790697891820955 us; error 67 us; skew 500 ppm
I20260929 16:04:51.821455 26575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:51 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697891, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:51.829895 26575 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:04:51.831852 26575 webserver.cc:460] Webserver started at http://127.20.121.190:33251/ using document root <none> and password file <none>
I20260929 16:04:51.832726 26575 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:51.832885 26575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:51.833246 26575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:51.836939 26575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "825d85e570e54c16a14e116b9910cf7c"
format_stamp: "Formatted at 2026-09-29 16:04:51 on dist-test-slave-x07d"
server_key: "f656751952b2c3106e79511923e8a94c"
server_key_iv: "475391d4bd93987aff9e75a3e1e68c20"
server_key_version: "encryptionkey@0"
I20260929 16:04:51.837904 26575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "825d85e570e54c16a14e116b9910cf7c"
format_stamp: "Formatted at 2026-09-29 16:04:51 on dist-test-slave-x07d"
server_key: "f656751952b2c3106e79511923e8a94c"
server_key_iv: "475391d4bd93987aff9e75a3e1e68c20"
server_key_version: "encryptionkey@0"
I20260929 16:04:51.843626 26575 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 16:04:51.847431 26590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:51.848907 26575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:04:51.849141 26575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "825d85e570e54c16a14e116b9910cf7c"
format_stamp: "Formatted at 2026-09-29 16:04:51 on dist-test-slave-x07d"
server_key: "f656751952b2c3106e79511923e8a94c"
server_key_iv: "475391d4bd93987aff9e75a3e1e68c20"
server_key_version: "encryptionkey@0"
I20260929 16:04:51.849370 26575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:51.869112 26575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:51.873728 26575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:51.874106 26575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:51.894119 26575 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:34291
I20260929 16:04:51.894147 26641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:34291 every 8 connection(s)
I20260929 16:04:51.895778 26575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:51.900549 26642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:51.900925 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 26575
I20260929 16:04:51.901180 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:51.901809 20966 external_mini_cluster.cc:1428] Setting key dc7c5f337898e93a44537b3309c28366
I20260929 16:04:51.916878 26642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c: Bootstrap starting.
Sep 29 16:04:51 dist-test-slave-x07d krb5kdc[26562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:51.920990 26642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:51.922540 26642 log.cc:824] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:51.925957 26642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c: No bootstrap required, opened a new log
I20260929 16:04:51.928638 26645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:51.904157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.20.121.190:34291)
0929 16:04:51.905111 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:51.905124 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:51.905176 (+ 52us) server_negotiation.cc:407] Connection header received
0929 16:04:51.907130 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:51.907160 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:51.907676 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:51.908090 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:51.910230 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:51.911750 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:51.913878 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:51.915002 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:51.915174 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:51.920449 (+ 5275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:51.920458 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:51.920475 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:51.920523 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:51.924263 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:51.925043 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:51.925050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:51.925057 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:51.925223 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:51.925611 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:51.925616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:51.925620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:51.926310 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:51.926639 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:51.927081 (+ 442us) server_negotiation.cc:299] Negotiation successful
0929 16:04:51.927535 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":472,"threads_started":1}
I20260929 16:04:51.935215 26642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825d85e570e54c16a14e116b9910cf7c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34291 } }
I20260929 16:04:51.935734 26642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:51.935871 26642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825d85e570e54c16a14e116b9910cf7c, State: Initialized, Role: FOLLOWER
I20260929 16:04:51.936654 26642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825d85e570e54c16a14e116b9910cf7c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34291 } }
I20260929 16:04:51.936858 26642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:51.936966 26642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:51.937098 26642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:51.939430 26642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825d85e570e54c16a14e116b9910cf7c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34291 } }
I20260929 16:04:51.939996 26642 leader_election.cc:302] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 825d85e570e54c16a14e116b9910cf7c; no voters:
I20260929 16:04:51.940591 26642 leader_election.cc:288] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:51.940728 26647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:51.942010 26647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [term 1 LEADER]: Becoming Leader. State: Replica: 825d85e570e54c16a14e116b9910cf7c, State: Running, Role: LEADER
I20260929 16:04:51.942647 26647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825d85e570e54c16a14e116b9910cf7c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34291 } }
I20260929 16:04:51.942888 26642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:51.945540 26649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 825d85e570e54c16a14e116b9910cf7c. Latest consensus state: current_term: 1 leader_uuid: "825d85e570e54c16a14e116b9910cf7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825d85e570e54c16a14e116b9910cf7c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34291 } } }
I20260929 16:04:51.945498 26648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "825d85e570e54c16a14e116b9910cf7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825d85e570e54c16a14e116b9910cf7c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34291 } } }
I20260929 16:04:51.946033 26648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:51.946653 26649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:51.946873 26654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:51.953505 26654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:51.964390 26654 catalog_manager.cc:1347] Generated new cluster ID: 7ea35a8de4ec4382b59ecd1de96162c6
I20260929 16:04:51.964537 26654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:51.979346 26654 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:51.981328 26654 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:51.998704 26654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 825d85e570e54c16a14e116b9910cf7c: Generated new TSK 0
I20260929 16:04:51.999879 26654 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:04:52.063578 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34291
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:52.421789 26668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:52.422309 26668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:52.422487 26668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:52.430562 26668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:52.430727 26668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:52.430914 26668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:52.441331 26668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:52.442885 26668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:52.444422 26668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:52.461020 26674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:52.461126 26673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:52.461076 26676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:52.467150 26668 server_base.cc:1034] running on GCE node
I20260929 16:04:52.468138 26668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:52.469372 26668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:52.470664 26668 hybrid_clock.cc:648] HybridClock initialized: now 1790697892470622 us; error 85 us; skew 500 ppm
I20260929 16:04:52.471088 26668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:52 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697892, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:52.475190 26668 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:04:52.477140 26668 webserver.cc:460] Webserver started at http://127.20.121.129:36021/ using document root <none> and password file <none>
I20260929 16:04:52.477984 26668 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:52.478230 26668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:52.478592 26668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:52.482096 26668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "008f0b2a40574910a5a3220b75461635"
format_stamp: "Formatted at 2026-09-29 16:04:52 on dist-test-slave-x07d"
server_key: "a6d6c5c33d5468b8a3d582db4015df00"
server_key_iv: "ee8ee196ee83e78520a42ec99ef75e59"
server_key_version: "encryptionkey@0"
I20260929 16:04:52.483126 26668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "008f0b2a40574910a5a3220b75461635"
format_stamp: "Formatted at 2026-09-29 16:04:52 on dist-test-slave-x07d"
server_key: "a6d6c5c33d5468b8a3d582db4015df00"
server_key_iv: "ee8ee196ee83e78520a42ec99ef75e59"
server_key_version: "encryptionkey@0"
I20260929 16:04:52.489540 26668 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260929 16:04:52.493731 26683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:52.495515 26668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:52.495743 26668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "008f0b2a40574910a5a3220b75461635"
format_stamp: "Formatted at 2026-09-29 16:04:52 on dist-test-slave-x07d"
server_key: "a6d6c5c33d5468b8a3d582db4015df00"
server_key_iv: "ee8ee196ee83e78520a42ec99ef75e59"
server_key_version: "encryptionkey@0"
I20260929 16:04:52.495999 26668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:52.525167 26668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:52.529925 26668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:52.530287 26668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:52.531301 26668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:52.532641 26668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:52.532716 26668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:52.532797 26668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:52.532841 26668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:52.571110 26668 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:46579
I20260929 16:04:52.571242 26795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:46579 every 8 connection(s)
I20260929 16:04:52.572605 26668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:52.578938 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 26668
I20260929 16:04:52.579236 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:52.580065 20966 external_mini_cluster.cc:1428] Setting key 8cfcefe9177e429289ffa8f16a3ff52a
Sep 29 16:04:52 dist-test-slave-x07d krb5kdc[26562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697892, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:52.598356 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:52.575638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:53577 (local address 127.20.121.190:34291)
0929 16:04:52.576085 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:52.576091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:52.577138 (+ 1047us) server_negotiation.cc:407] Connection header received
0929 16:04:52.579271 (+ 2133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:52.579278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:52.579382 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:52.579544 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:52.581448 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:52.582571 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:52.585543 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:52.585946 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:52.586220 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:52.591101 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:52.591107 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:52.591110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:52.591150 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:52.594164 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:52.595078 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:52.595084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:52.595088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:52.595222 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:52.595725 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:52.595733 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:52.595737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:52.596038 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:52.596486 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:52.597591 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0929 16:04:52.597855 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":238,"threads_started":1}
I20260929 16:04:52.599661 26799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:52.576151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34291 (local address 127.20.121.129:53577)
0929 16:04:52.576944 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:52.576994 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:52.578833 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:52.579803 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:52.579814 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:52.580452 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:52.581217 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:52.581239 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:52.582774 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:52.582782 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:52.585302 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:52.585324 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:52.586345 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:52.586355 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:52.586710 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:52.587559 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:52.587595 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:52.590791 (+ 3196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:52.594383 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:52.594393 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:52.594409 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:52.594915 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:52.595362 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:52.595372 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:52.595374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:52.595587 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:52.596479 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:52.596490 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:52.596910 (+ 420us) client_negotiation.cc:769] Sending connection context
0929 16:04:52.597457 (+ 547us) client_negotiation.cc:240] Negotiation successful
0929 16:04:52.598140 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":362,"threads_started":1}
I20260929 16:04:52.601784 26796 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34291
I20260929 16:04:52.602218 26796 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:04:52.603044 26796 heartbeater.cc:511] Master 127.20.121.190:34291 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:04:52.606294 26607 ts_manager.cc:194] Registered new tserver with Master: 008f0b2a40574910a5a3220b75461635 (127.20.121.129:46579)
I20260929 16:04:52.607914 26607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:53577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:04:52.639935 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34291
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:52.992092 26802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:52.992414 26802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:52.992523 26802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:53.001186 26802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:53.001343 26802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:53.001525 26802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:53.013127 26802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:53.014882 26802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:53.016458 26802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:53.032860 26807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:53.032879 26808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:53.032820 26810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:53.036259 26802 server_base.cc:1034] running on GCE node
I20260929 16:04:53.037180 26802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:53.038542 26802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:53.039836 26802 hybrid_clock.cc:648] HybridClock initialized: now 1790697893039794 us; error 72 us; skew 500 ppm
I20260929 16:04:53.040238 26802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:53.044430 26802 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:04:53.046447 26802 webserver.cc:460] Webserver started at http://127.20.121.130:39007/ using document root <none> and password file <none>
I20260929 16:04:53.047298 26802 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:53.047422 26802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:53.047794 26802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:53.051129 26802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "bc0d2ff9dd8340738c0381a7e311ea9c"
format_stamp: "Formatted at 2026-09-29 16:04:53 on dist-test-slave-x07d"
server_key: "51061ad4d8faf8034a46cc4054bba44e"
server_key_iv: "f29e02bdfd9990f00edafe0b86943b0d"
server_key_version: "encryptionkey@0"
I20260929 16:04:53.052104 26802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "bc0d2ff9dd8340738c0381a7e311ea9c"
format_stamp: "Formatted at 2026-09-29 16:04:53 on dist-test-slave-x07d"
server_key: "51061ad4d8faf8034a46cc4054bba44e"
server_key_iv: "f29e02bdfd9990f00edafe0b86943b0d"
server_key_version: "encryptionkey@0"
I20260929 16:04:53.058151 26802 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260929 16:04:53.062184 26817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:53.063825 26802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:53.064103 26802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "bc0d2ff9dd8340738c0381a7e311ea9c"
format_stamp: "Formatted at 2026-09-29 16:04:53 on dist-test-slave-x07d"
server_key: "51061ad4d8faf8034a46cc4054bba44e"
server_key_iv: "f29e02bdfd9990f00edafe0b86943b0d"
server_key_version: "encryptionkey@0"
I20260929 16:04:53.064340 26802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:53.078380 26802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:53.083501 26802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:53.083858 26802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:53.084925 26802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:53.086486 26802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:53.086609 26802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:53.086762 26802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:53.086860 26802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:53.120914 26802 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:35393
I20260929 16:04:53.120960 26929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:35393 every 8 connection(s)
I20260929 16:04:53.122359 26802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:53.123684 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 26802
I20260929 16:04:53.123886 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:53.124264 20966 external_mini_cluster.cc:1428] Setting key 7b2c30fef2d0d229606ce66a7e918e64
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
I20260929 16:04:53.149035 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:53.125467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:53253 (local address 127.20.121.190:34291)
0929 16:04:53.126077 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:53.126085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:53.127415 (+ 1330us) server_negotiation.cc:407] Connection header received
0929 16:04:53.129473 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:53.129479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:53.129582 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:53.129730 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:53.131656 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.133022 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.135525 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.136371 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.136545 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:53.141002 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:53.141011 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:53.141020 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:53.141074 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:53.144733 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.146000 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.146052 (+ 52us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.146057 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.146218 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.146860 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.146865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.146933 (+ 68us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.147327 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:53.147519 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:53.148388 (+ 869us) server_negotiation.cc:299] Negotiation successful
0929 16:04:53.148658 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":249,"threads_started":1}
I20260929 16:04:53.150269 26933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:53.126275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34291 (local address 127.20.121.130:53253)
0929 16:04:53.127240 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:53.127301 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:53.129158 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:53.130058 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:53.130076 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:53.130741 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:53.131420 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:53.131441 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.133301 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.133307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:53.135298 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:53.135318 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.136704 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.136711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:53.136968 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:53.137857 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:53.137882 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:53.140660 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:53.144959 (+ 4299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:53.144973 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:53.145000 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:53.145758 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:53.146369 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:53.146378 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:53.146383 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:53.146692 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:53.147502 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:53.147509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:53.147950 (+ 441us) client_negotiation.cc:769] Sending connection context
0929 16:04:53.148290 (+ 340us) client_negotiation.cc:240] Negotiation successful
0929 16:04:53.148865 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":357,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
I20260929 16:04:53.152271 26930 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34291
I20260929 16:04:53.152644 26930 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:53.153666 26930 heartbeater.cc:511] Master 127.20.121.190:34291 requested a full tablet report, sending...
I20260929 16:04:53.155607 26607 ts_manager.cc:194] Registered new tserver with Master: bc0d2ff9dd8340738c0381a7e311ea9c (127.20.121.130:35393)
I20260929 16:04:53.156463 26607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:53253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:04:53.183137 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34291
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:53.522791 26936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:53.523089 26936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:53.523243 26936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:53.530726 26936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:53.530845 26936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:53.531029 26936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:53.541082 26936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:53.542567 26936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:53.543978 26936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:53.557763 26944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:53.557680 26942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:53.558250 26941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:53.559247 26936 server_base.cc:1034] running on GCE node
I20260929 16:04:53.559969 26936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:53.561086 26936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:53.562330 26936 hybrid_clock.cc:648] HybridClock initialized: now 1790697893562259 us; error 51 us; skew 500 ppm
I20260929 16:04:53.562700 26936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:53.566133 26936 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:04:53.567690 26936 webserver.cc:460] Webserver started at http://127.20.121.131:42015/ using document root <none> and password file <none>
I20260929 16:04:53.568429 26936 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:53.568531 26936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:53.568779 26936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:53.571873 26936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "227e3953cd67464d8b0a2cc8f7fd98b2"
format_stamp: "Formatted at 2026-09-29 16:04:53 on dist-test-slave-x07d"
server_key: "af40b69553c28f71674129c51f1b7452"
server_key_iv: "79233ba38e449cc10af479efe89f8843"
server_key_version: "encryptionkey@0"
I20260929 16:04:53.572719 26936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "227e3953cd67464d8b0a2cc8f7fd98b2"
format_stamp: "Formatted at 2026-09-29 16:04:53 on dist-test-slave-x07d"
server_key: "af40b69553c28f71674129c51f1b7452"
server_key_iv: "79233ba38e449cc10af479efe89f8843"
server_key_version: "encryptionkey@0"
W20260929 16:04:53.574651 26792 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.20.121.129:46579
I20260929 16:04:53.578411 26936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 16:04:53.582170 26951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:53.583804 26936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:04:53.584003 26936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "227e3953cd67464d8b0a2cc8f7fd98b2"
format_stamp: "Formatted at 2026-09-29 16:04:53 on dist-test-slave-x07d"
server_key: "af40b69553c28f71674129c51f1b7452"
server_key_iv: "79233ba38e449cc10af479efe89f8843"
server_key_version: "encryptionkey@0"
I20260929 16:04:53.584223 26936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:53.607074 26936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:53.612335 26936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:53.612679 26796 heartbeater.cc:503] Master 127.20.121.190:34291 was elected leader, sending a full tablet report...
I20260929 16:04:53.612738 26936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:53.614046 26936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:53.615638 26936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:53.615728 26936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:53.615832 26936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:53.615885 26936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:53.655259 26936 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:37831
I20260929 16:04:53.655368 27063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:37831 every 8 connection(s)
I20260929 16:04:53.656725 26936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:53.664994 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 26936
I20260929 16:04:53.665238 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:53.665730 20966 external_mini_cluster.cc:1428] Setting key 856a9cbf79e8a55b4d6b03ef35315e78
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:04:53.682932 27066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:53.659852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:48591 (local address 127.20.121.190:34291)
0929 16:04:53.660351 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:53.660359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:53.661610 (+ 1251us) server_negotiation.cc:407] Connection header received
0929 16:04:53.663704 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:53.663709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:53.663793 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:53.663902 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:53.666052 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.667028 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.669808 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.670293 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.670425 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:53.674944 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:53.674951 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:53.674955 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:53.675000 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:53.678183 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.679281 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.679287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.679289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.679435 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.680158 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.680164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.680167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.680578 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:53.680780 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:53.682234 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0929 16:04:53.682524 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":240,"threads_started":1}
I20260929 16:04:53.684628 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:53.660553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34291 (local address 127.20.121.131:48591)
0929 16:04:53.661406 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:53.661457 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:53.663285 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:53.664236 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:53.664246 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:53.664893 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:53.665741 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:53.665763 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.667338 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.667345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:53.669649 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:53.669666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.670564 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.670570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:53.670796 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:53.671721 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:53.671755 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:53.674562 (+ 2807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:53.678412 (+ 3850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:53.678421 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:53.678438 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:53.679078 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:53.679655 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:53.679659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:53.679662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:53.679948 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:53.680809 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:53.680822 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:53.681559 (+ 737us) client_negotiation.cc:769] Sending connection context
0929 16:04:53.682039 (+ 480us) client_negotiation.cc:240] Negotiation successful
0929 16:04:53.682757 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":352,"threads_started":1}
I20260929 16:04:53.686714 27064 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34291
I20260929 16:04:53.687094 27064 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:53.688180 27064 heartbeater.cc:511] Master 127.20.121.190:34291 requested a full tablet report, sending...
I20260929 16:04:53.690304 26607 ts_manager.cc:194] Registered new tserver with Master: 227e3953cd67464d8b0a2cc8f7fd98b2 (127.20.121.131:37831)
I20260929 16:04:53.691545 26607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:48591
I20260929 16:04:53.696161 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/test-admin.keytab.
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:53.727730 20966 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 16:04:53.733314 27072 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:04:53.730772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49096 (local address 127.20.121.129:46579)
0929 16:04:53.731425 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:53.731433 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:53.731456 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:04:53.731513 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:53.731519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:53.731727 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:53.731892 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:53.732872 (+ 980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49096: BlockingRecv error: recv got EOF from 127.0.0.1:49096 (error 108)
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":344,"threads_started":1}
W20260929 16:04:53.734200 20966 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5cc)
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:53.735880 20966 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:53.737298 20966 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:53.755038 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:53.739162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49110 (local address 127.20.121.129:46579)
0929 16:04:53.739474 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:53.739481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:53.739671 (+ 190us) server_negotiation.cc:407] Connection header received
0929 16:04:53.739803 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:53.739808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:53.739875 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:53.739978 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:53.740961 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.742008 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.743495 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.744108 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.744235 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:53.748081 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:53.748095 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:53.748113 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:53.748161 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:53.751707 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.752631 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.752639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.752642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.752776 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.753217 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.753225 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.753228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.753638 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:53.753977 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:53.754414 (+ 437us) server_negotiation.cc:299] Negotiation successful
0929 16:04:53.754703 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260929 16:04:53.761482 27072 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:04:53.759793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.20.121.129:46579)
0929 16:04:53.760007 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:53.760013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:53.760212 (+ 199us) server_negotiation.cc:407] Connection header received
0929 16:04:53.760349 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:53.760353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:53.760411 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:53.760492 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:53.761269 (+ 777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49114: BlockingRecv error: recv got EOF from 127.0.0.1:49114 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260929 16:04:53.761793 20966 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5cc' of type FILE
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:53.763630 20966 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:53.765221 20966 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:53.781697 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:53.767047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.20.121.129:46579)
0929 16:04:53.767320 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:53.767327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:53.767673 (+ 346us) server_negotiation.cc:407] Connection header received
0929 16:04:53.767850 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:53.767856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:53.767925 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:53.768042 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:53.769138 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.769901 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.771364 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.771852 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.772023 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:53.776292 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:53.776298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:53.776302 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:53.776373 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:53.779172 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.779868 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.779872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.779875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.780068 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.780416 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.780421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.780424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.780710 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:53.780974 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:53.781226 (+ 252us) server_negotiation.cc:299] Negotiation successful
0929 16:04:53.781422 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
W20260929 16:04:53.786933 27072 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:04:53.784851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.20.121.129:46579)
0929 16:04:53.785162 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:53.785169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:53.785374 (+ 205us) server_negotiation.cc:407] Connection header received
0929 16:04:53.785613 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:53.785617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:53.785690 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:53.785806 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:53.786663 (+ 857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49118: BlockingRecv error: recv got EOF from 127.0.0.1:49118 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260929 16:04:53.787176 20966 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790697841464038-20966-0/krb5kdc/krb5cc' of type FILE
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:53.788841 20966 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:53.790305 20966 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:04:53.805316 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:53.791663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49124 (local address 127.20.121.129:46579)
0929 16:04:53.791890 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:53.791898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:53.792257 (+ 359us) server_negotiation.cc:407] Connection header received
0929 16:04:53.792463 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:53.792467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:53.792533 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:53.792652 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:53.793531 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.794317 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.795514 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.795916 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.796042 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:53.799639 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:53.799644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:53.799647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:53.799678 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:53.802847 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.803452 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.803458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.803462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.803602 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.803959 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.803964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.803967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.804250 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:53.804560 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:53.804783 (+ 223us) server_negotiation.cc:299] Negotiation successful
0929 16:04:53.804998 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 16:04:53.807104 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 26668
I20260929 16:04:53.819957 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:53.807956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49132 (local address 127.20.121.129:46579)
0929 16:04:53.808187 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:53.808192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:53.808415 (+ 223us) server_negotiation.cc:407] Connection header received
0929 16:04:53.808574 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:53.808582 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:53.808659 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:53.808761 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:53.809784 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.810488 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.811806 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.812188 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.812276 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:53.814554 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:53.814558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:53.814562 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:53.814590 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:53.817153 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.817868 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.817873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.817875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.818174 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.818573 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.818577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.818579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.818835 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:53.819034 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:53.819373 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 16:04:53.819539 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260929 16:04:53.820673 26791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:53.906220 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 26668
I20260929 16:04:53.924459 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 26802
Sep 29 16:04:53 dist-test-slave-x07d krb5kdc[26562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:04:53.943862 27085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:53.925432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60232 (local address 127.20.121.130:35393)
0929 16:04:53.925988 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:53.925995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:53.926079 (+ 84us) server_negotiation.cc:407] Connection header received
0929 16:04:53.926151 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:53.926157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:53.926428 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:53.926575 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:53.927757 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.928811 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.930445 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:53.931060 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:53.931189 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:53.935970 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:53.935989 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:53.936008 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:53.936055 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:53.939715 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.940917 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.940930 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.940934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.941283 (+ 349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:53.941739 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:53.941742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:53.941745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:53.942257 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:53.942512 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:53.943164 (+ 652us) server_negotiation.cc:299] Negotiation successful
0929 16:04:53.943520 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":289,"threads_started":1}
I20260929 16:04:53.945531 26925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:54.032819 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 26802
I20260929 16:04:54.049685 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 26936
Sep 29 16:04:54 dist-test-slave-x07d krb5kdc[26562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:04:54.067278 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:54.050926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.20.121.131:37831)
0929 16:04:54.051625 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:54.051633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:54.051665 (+ 32us) server_negotiation.cc:407] Connection header received
0929 16:04:54.051725 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:54.051732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:54.051934 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:54.052081 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:54.053384 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:54.054712 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:54.055977 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:54.056533 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:54.056636 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:54.060910 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:54.060923 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:54.060947 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:54.060990 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:54.064214 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:54.065149 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:54.065159 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:54.065162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:54.065332 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:54.065787 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:54.065790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:54.065793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:54.066272 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:54.066582 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:54.066777 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 16:04:54.066990 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":385,"threads_started":1}
I20260929 16:04:54.068717 27059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:54.159722 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 26936
I20260929 16:04:54.178232 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 26575
I20260929 16:04:54.179765 26637 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:54.286562 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 26575
2026-09-29T16:04:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4240 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:04:54 dist-test-slave-x07d krb5kdc[27092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:04:54 dist-test-slave-x07d krb5kdc[27092](info): set up 2 sockets
Sep 29 16:04:54 dist-test-slave-x07d krb5kdc[27092](info): commencing operation
krb5kdc: starting...
W20260929 16:04:55.455014 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:04:55 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:04:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:04:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.190.keytab.
Entry for principal oryx/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.190.keytab.
I20260929 16:04:55.683866 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:43697
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.190.keytab
--principal=oryx/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:43697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:56.022197 27105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:56.022531 27105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:56.022624 27105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:56.031242 27105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:56.031399 27105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:56.031471 27105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:56.031629 27105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:56.031744 27105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:56.043314 27105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:43697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.190.keytab
--principal=oryx/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:43697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:56.045253 27105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:56.046813 27105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:56.061092 27110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:56.060997 27113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:56.061067 27111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:56.062070 27105 server_base.cc:1034] running on GCE node
I20260929 16:04:56.063037 27105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:56.064723 27105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:56.065951 27105 hybrid_clock.cc:648] HybridClock initialized: now 1790697896065942 us; error 78 us; skew 500 ppm
I20260929 16:04:56.066407 27105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:56 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697896, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:56.069908 27105 init.cc:377] Logged in from keytab as oryx/127.20.121.190@KRBTEST.COM (short username oryx)
I20260929 16:04:56.071605 27105 webserver.cc:460] Webserver started at http://127.20.121.190:43743/ using document root <none> and password file <none>
I20260929 16:04:56.072355 27105 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:56.072496 27105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:56.072834 27105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:56.076125 27105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "7adf02d0479447608d50d296296b2eaa"
format_stamp: "Formatted at 2026-09-29 16:04:56 on dist-test-slave-x07d"
server_key: "08d6685739004c98076dca1daf533045"
server_key_iv: "4b7a02c760eda4cc7131de9b60fc5cb9"
server_key_version: "encryptionkey@0"
I20260929 16:04:56.076951 27105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "7adf02d0479447608d50d296296b2eaa"
format_stamp: "Formatted at 2026-09-29 16:04:56 on dist-test-slave-x07d"
server_key: "08d6685739004c98076dca1daf533045"
server_key_iv: "4b7a02c760eda4cc7131de9b60fc5cb9"
server_key_version: "encryptionkey@0"
I20260929 16:04:56.082890 27105 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260929 16:04:56.087296 27120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:56.088860 27105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:04:56.089182 27105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "7adf02d0479447608d50d296296b2eaa"
format_stamp: "Formatted at 2026-09-29 16:04:56 on dist-test-slave-x07d"
server_key: "08d6685739004c98076dca1daf533045"
server_key_iv: "4b7a02c760eda4cc7131de9b60fc5cb9"
server_key_version: "encryptionkey@0"
I20260929 16:04:56.089448 27105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:56.111351 27105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:56.116569 27105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:56.116925 27105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:56.138873 27105 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:43697
I20260929 16:04:56.138922 27171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:43697 every 8 connection(s)
I20260929 16:04:56.140408 27105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:04:56.144416 27172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:56.146756 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 27105
I20260929 16:04:56.147017 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:04:56.147899 20966 external_mini_cluster.cc:1428] Setting key 22fc427d132a66b22d47e03785791a6f
I20260929 16:04:56.158803 27172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa: Bootstrap starting.
I20260929 16:04:56.162398 27172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:56.163607 27172 log.cc:824] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa: Log is configured to *not* fsync() on all Append() calls
Sep 29 16:04:56 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:04:56.167222 27172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa: No bootstrap required, opened a new log
I20260929 16:04:56.175215 27175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:56.152188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34218 (local address 127.20.121.190:43697)
0929 16:04:56.153114 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:56.153132 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:56.153189 (+ 57us) server_negotiation.cc:407] Connection header received
0929 16:04:56.155093 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:56.155120 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:56.155589 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:56.156088 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:56.157731 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:56.159264 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:56.160736 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:56.161476 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:56.161708 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:56.166171 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:56.166180 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:56.166197 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:56.166243 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:56.169887 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:56.170807 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:56.170822 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:56.170829 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:56.171035 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:56.171477 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:56.171482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:56.171485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:56.172175 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:56.172545 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:56.173196 (+ 651us) server_negotiation.cc:299] Negotiation successful
0929 16:04:56.173821 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":383,"threads_started":1}
I20260929 16:04:56.176050 27172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } }
I20260929 16:04:56.176467 27172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:56.176654 27172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7adf02d0479447608d50d296296b2eaa, State: Initialized, Role: FOLLOWER
I20260929 16:04:56.177363 27172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } }
I20260929 16:04:56.177680 27172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:04:56.177845 27172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:04:56.178069 27172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:04:56.180600 27172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } }
I20260929 16:04:56.181196 27172 leader_election.cc:302] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7adf02d0479447608d50d296296b2eaa; no voters:
I20260929 16:04:56.181890 27172 leader_election.cc:288] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:04:56.182156 27177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:04:56.183280 27177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 1 LEADER]: Becoming Leader. State: Replica: 7adf02d0479447608d50d296296b2eaa, State: Running, Role: LEADER
I20260929 16:04:56.183966 27177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } }
I20260929 16:04:56.184201 27172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:56.186940 27179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7adf02d0479447608d50d296296b2eaa. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } }
I20260929 16:04:56.187309 27179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:56.187942 27178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } }
I20260929 16:04:56.188292 27178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:56.188328 27184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:04:56.194896 27184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:04:56.205276 27184 catalog_manager.cc:1347] Generated new cluster ID: 635f15c821ad4c669fd2eadd14d2dfb2
I20260929 16:04:56.205420 27184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:04:56.218781 27184 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:04:56.221278 27184 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:04:56.233861 27184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa: Generated new TSK 0
I20260929 16:04:56.234974 27184 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab.
Entry for principal oryx/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab.
I20260929 16:04:56.307140 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:43697
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab
--principal=oryx/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:56.649678 27198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:56.649987 27198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:56.650182 27198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:56.657781 27198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:56.657922 27198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:56.658126 27198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:56.668417 27198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab
--principal=oryx/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:43697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:56.669752 27198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:56.671030 27198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:56.685366 27204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:56.685449 27206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:56.685400 27203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:56.685892 27198 server_base.cc:1034] running on GCE node
I20260929 16:04:56.686779 27198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:56.687740 27198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:56.688953 27198 hybrid_clock.cc:648] HybridClock initialized: now 1790697896688892 us; error 47 us; skew 500 ppm
I20260929 16:04:56.689349 27198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:56 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697896, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:56.693199 27198 init.cc:377] Logged in from keytab as oryx/127.20.121.129@KRBTEST.COM (short username oryx)
I20260929 16:04:56.694970 27198 webserver.cc:460] Webserver started at http://127.20.121.129:36407/ using document root <none> and password file <none>
I20260929 16:04:56.695679 27198 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:56.695788 27198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:56.696054 27198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:56.699331 27198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0"
format_stamp: "Formatted at 2026-09-29 16:04:56 on dist-test-slave-x07d"
server_key: "9f188ac29c78bbc72acf54e8885d49e7"
server_key_iv: "f6b69e1974d4fd06a5268509647a6082"
server_key_version: "encryptionkey@0"
I20260929 16:04:56.700177 27198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0"
format_stamp: "Formatted at 2026-09-29 16:04:56 on dist-test-slave-x07d"
server_key: "9f188ac29c78bbc72acf54e8885d49e7"
server_key_iv: "f6b69e1974d4fd06a5268509647a6082"
server_key_version: "encryptionkey@0"
I20260929 16:04:56.705796 27198 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 16:04:56.709681 27213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:56.711342 27198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 16:04:56.711545 27198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0"
format_stamp: "Formatted at 2026-09-29 16:04:56 on dist-test-slave-x07d"
server_key: "9f188ac29c78bbc72acf54e8885d49e7"
server_key_iv: "f6b69e1974d4fd06a5268509647a6082"
server_key_version: "encryptionkey@0"
I20260929 16:04:56.711836 27198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:56.729417 27198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:56.734612 27198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:56.734915 27198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:56.735893 27198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:56.737392 27198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:56.737498 27198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:56.737646 27198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:56.737725 27198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:56.772941 27198 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:44627
I20260929 16:04:56.772943 27325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:44627 every 8 connection(s)
I20260929 16:04:56.774302 27198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:04:56.783440 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 27198
I20260929 16:04:56.783684 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:04:56.784353 20966 external_mini_cluster.cc:1428] Setting key b532a0e8b65291ed00e57ec2a27763cd
Sep 29 16:04:56 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697896, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:04:56.796762 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:56.776953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:56329 (local address 127.20.121.190:43697)
0929 16:04:56.777396 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:56.777400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:56.778621 (+ 1221us) server_negotiation.cc:407] Connection header received
0929 16:04:56.780471 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:56.780477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:56.780580 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:56.780701 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:56.782300 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:56.783070 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:56.785019 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:56.785544 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:56.785665 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:56.789962 (+ 4297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:56.789968 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:56.789971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:56.790050 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:56.793066 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:56.794212 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:56.794217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:56.794220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:56.794344 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:56.794912 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:56.794917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:56.794919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:56.795187 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:56.795341 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:56.796169 (+ 828us) server_negotiation.cc:299] Negotiation successful
0929 16:04:56.796399 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":220,"threads_started":1}
I20260929 16:04:56.797761 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:56.777496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.129:56329)
0929 16:04:56.778448 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:56.778493 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:56.780188 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:56.780944 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:56.780953 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:56.781476 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:56.782126 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:56.782146 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:56.783240 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:56.783250 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:56.784819 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:56.784832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:56.785783 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:56.785790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:56.786155 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:56.786881 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:56.786911 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:56.789709 (+ 2798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:56.793226 (+ 3517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:56.793238 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:56.793282 (+ 44us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:56.793975 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:56.794552 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:56.794556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:56.794559 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:56.794751 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:56.795330 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:56.795338 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:56.795723 (+ 385us) client_negotiation.cc:769] Sending connection context
0929 16:04:56.795999 (+ 276us) client_negotiation.cc:240] Negotiation successful
0929 16:04:56.796575 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":383,"threads_started":1}
I20260929 16:04:56.799517 27326 heartbeater.cc:347] Connected to a master server at 127.20.121.190:43697
I20260929 16:04:56.799839 27326 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:56.800601 27326 heartbeater.cc:511] Master 127.20.121.190:43697 requested a full tablet report, sending...
I20260929 16:04:56.802793 27137 ts_manager.cc:194] Registered new tserver with Master: 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627)
I20260929 16:04:56.804250 27137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:56329
WARNING: no policy specified for oryx/127.20.121.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab.
Entry for principal oryx/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab.
I20260929 16:04:56.834767 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:43697
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab
--principal=oryx/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:57.183584 27332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:57.183903 27332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:57.184078 27332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:57.191787 27332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:57.191964 27332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:57.192138 27332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:04:57.202975 27332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab
--principal=oryx/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:43697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:57.204270 27332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:57.205585 27332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:57.219890 27338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:57.219929 27337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:57.219826 27340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:57.220955 27332 server_base.cc:1034] running on GCE node
I20260929 16:04:57.221657 27332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:57.222723 27332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:57.223963 27332 hybrid_clock.cc:648] HybridClock initialized: now 1790697897223889 us; error 71 us; skew 500 ppm
I20260929 16:04:57.224359 27332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:57 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697897, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:57.237008 27332 init.cc:377] Logged in from keytab as oryx/127.20.121.130@KRBTEST.COM (short username oryx)
I20260929 16:04:57.238886 27332 webserver.cc:460] Webserver started at http://127.20.121.130:42903/ using document root <none> and password file <none>
I20260929 16:04:57.239706 27332 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:57.239876 27332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:57.240198 27332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:57.243325 27332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "5188495bc43c49e48ebdf870aa0b5d80"
format_stamp: "Formatted at 2026-09-29 16:04:57 on dist-test-slave-x07d"
server_key: "0ed91f2cdf57ce62cc0059abec526760"
server_key_iv: "6a83a79e1e31520b2074e500d56ce888"
server_key_version: "encryptionkey@0"
I20260929 16:04:57.244179 27332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "5188495bc43c49e48ebdf870aa0b5d80"
format_stamp: "Formatted at 2026-09-29 16:04:57 on dist-test-slave-x07d"
server_key: "0ed91f2cdf57ce62cc0059abec526760"
server_key_iv: "6a83a79e1e31520b2074e500d56ce888"
server_key_version: "encryptionkey@0"
I20260929 16:04:57.249743 27332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 16:04:57.253705 27347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:57.255445 27332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:04:57.255683 27332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "5188495bc43c49e48ebdf870aa0b5d80"
format_stamp: "Formatted at 2026-09-29 16:04:57 on dist-test-slave-x07d"
server_key: "0ed91f2cdf57ce62cc0059abec526760"
server_key_iv: "6a83a79e1e31520b2074e500d56ce888"
server_key_version: "encryptionkey@0"
I20260929 16:04:57.255920 27332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:57.273811 27332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:57.278635 27332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:57.278949 27332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:57.279994 27332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:57.281452 27332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:57.281554 27332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:57.281705 27332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:57.281778 27332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:57.316934 27332 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:37735
I20260929 16:04:57.316968 27459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:37735 every 8 connection(s)
I20260929 16:04:57.318261 27332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:04:57.326894 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 27332
I20260929 16:04:57.327108 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:04:57.327652 20966 external_mini_cluster.cc:1428] Setting key 24f33506f57de448e62a7381c6784d4a
Sep 29 16:04:57 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697897, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:04:57.340260 27462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:57.320877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:59555 (local address 127.20.121.190:43697)
0929 16:04:57.321329 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:57.321333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:57.322519 (+ 1186us) server_negotiation.cc:407] Connection header received
0929 16:04:57.324289 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:57.324294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:57.324363 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:57.324464 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:57.326185 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:57.327289 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:57.329283 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:57.329935 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:57.330130 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:57.334446 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:57.334451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:57.334454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:57.334488 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:57.337096 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:57.337867 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:57.337873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:57.337878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:57.338088 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:57.338512 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:57.338517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:57.338520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:57.338789 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:57.338943 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:57.339665 (+ 722us) server_negotiation.cc:299] Negotiation successful
0929 16:04:57.339881 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":237,"threads_started":1}
I20260929 16:04:57.341276 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:57.321475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.130:59555)
0929 16:04:57.322353 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:57.322392 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:57.324000 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:57.324711 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:57.324719 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:57.325301 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:57.325938 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:57.325958 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:57.327474 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:57.327480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:57.329135 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:57.329148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:57.330239 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:57.330246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:57.330602 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:57.331378 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:57.331422 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:57.334201 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:57.337268 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:57.337278 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:57.337297 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:57.337735 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:57.338198 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:57.338204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:57.338208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:57.338396 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:57.338914 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:57.338922 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:57.339290 (+ 368us) client_negotiation.cc:769] Sending connection context
0929 16:04:57.339572 (+ 282us) client_negotiation.cc:240] Negotiation successful
0929 16:04:57.340125 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":362,"threads_started":1}
I20260929 16:04:57.342803 27460 heartbeater.cc:347] Connected to a master server at 127.20.121.190:43697
I20260929 16:04:57.343130 27460 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:57.343885 27460 heartbeater.cc:511] Master 127.20.121.190:43697 requested a full tablet report, sending...
I20260929 16:04:57.345553 27137 ts_manager.cc:194] Registered new tserver with Master: 5188495bc43c49e48ebdf870aa0b5d80 (127.20.121.130:37735)
I20260929 16:04:57.346890 27137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:59555
WARNING: no policy specified for oryx/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab.
Entry for principal oryx/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab.
I20260929 16:04:57.378376 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:43697
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab
--principal=oryx/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:57.726833 27466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:57.727180 27466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:57.727276 27466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:57.735165 27466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:57.735294 27466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:57.735412 27466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:04:57.746079 27466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab
--principal=oryx/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:43697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:57.747576 27466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:57.749097 27466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:57.763513 27471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:57.763583 27472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:57.763530 27474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:57.764654 27466 server_base.cc:1034] running on GCE node
I20260929 16:04:57.765342 27466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:57.766507 27466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:57.767771 27466 hybrid_clock.cc:648] HybridClock initialized: now 1790697897767719 us; error 49 us; skew 500 ppm
I20260929 16:04:57.768214 27466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:57 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697897, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:57.771622 27466 init.cc:377] Logged in from keytab as oryx/127.20.121.131@KRBTEST.COM (short username oryx)
I20260929 16:04:57.773384 27466 webserver.cc:460] Webserver started at http://127.20.121.131:33773/ using document root <none> and password file <none>
I20260929 16:04:57.774155 27466 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:57.774303 27466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:57.774609 27466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:57.777518 27466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "37e9aca105a7457c8459e7c732a12f20"
format_stamp: "Formatted at 2026-09-29 16:04:57 on dist-test-slave-x07d"
server_key: "9bb47fa958159ca462e1fa4e8ed8c674"
server_key_iv: "46fd00e2d3f3501b7bcdc457a4ead03a"
server_key_version: "encryptionkey@0"
I20260929 16:04:57.778393 27466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "37e9aca105a7457c8459e7c732a12f20"
format_stamp: "Formatted at 2026-09-29 16:04:57 on dist-test-slave-x07d"
server_key: "9bb47fa958159ca462e1fa4e8ed8c674"
server_key_iv: "46fd00e2d3f3501b7bcdc457a4ead03a"
server_key_version: "encryptionkey@0"
I20260929 16:04:57.784134 27466 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 16:04:57.788120 27481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:57.789729 27466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:04:57.789917 27466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "37e9aca105a7457c8459e7c732a12f20"
format_stamp: "Formatted at 2026-09-29 16:04:57 on dist-test-slave-x07d"
server_key: "9bb47fa958159ca462e1fa4e8ed8c674"
server_key_iv: "46fd00e2d3f3501b7bcdc457a4ead03a"
server_key_version: "encryptionkey@0"
I20260929 16:04:57.790213 27466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:57.806440 27466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:57.807511 27326 heartbeater.cc:503] Master 127.20.121.190:43697 was elected leader, sending a full tablet report...
I20260929 16:04:57.811295 27466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:57.811596 27466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:57.812594 27466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:04:57.813992 27466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:04:57.814131 27466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:57.814281 27466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:04:57.814371 27466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:57.849488 27466 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:33721
I20260929 16:04:57.849529 27593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:33721 every 8 connection(s)
I20260929 16:04:57.850932 27466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:04:57.860747 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 27466
I20260929 16:04:57.860970 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:04:57.861389 20966 external_mini_cluster.cc:1428] Setting key b19e5583723fb68e48cbd064a4f2ec5e
Sep 29 16:04:57 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697897, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:04:57.872467 27596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:57.853785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:39635 (local address 127.20.121.190:43697)
0929 16:04:57.854299 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:57.854304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:57.855319 (+ 1015us) server_negotiation.cc:407] Connection header received
0929 16:04:57.857143 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:57.857149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:57.857250 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:57.857374 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:57.859180 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:57.860078 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:57.862082 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:57.862510 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:57.862653 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:57.866275 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:57.866280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:57.866284 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:57.866319 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:57.869225 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:57.870156 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:57.870161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:57.870164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:57.870293 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:57.870768 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:57.870772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:57.870774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:57.871088 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:57.871220 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:57.872005 (+ 785us) server_negotiation.cc:299] Negotiation successful
0929 16:04:57.872177 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":305,"threads_started":1}
I20260929 16:04:57.873405 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:57.854339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.131:39635)
0929 16:04:57.855137 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:57.855184 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:57.856853 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:57.857629 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:57.857638 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:57.858286 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:57.858881 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:57.858899 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:57.860328 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:57.860333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:57.861902 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:57.861916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:57.862769 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:57.862776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:57.863026 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:57.863765 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:57.863787 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:57.865999 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:57.869400 (+ 3401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:57.869409 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:57.869424 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:57.869888 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:57.870393 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:57.870396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:57.870399 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:57.870619 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:57.871208 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:57.871215 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:57.871587 (+ 372us) client_negotiation.cc:769] Sending connection context
0929 16:04:57.871819 (+ 232us) client_negotiation.cc:240] Negotiation successful
0929 16:04:57.872218 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":314,"threads_started":1}
I20260929 16:04:57.874871 27594 heartbeater.cc:347] Connected to a master server at 127.20.121.190:43697
I20260929 16:04:57.875249 27594 heartbeater.cc:464] Registering TS with master...
I20260929 16:04:57.876258 27594 heartbeater.cc:511] Master 127.20.121.190:43697 requested a full tablet report, sending...
I20260929 16:04:57.877777 27137 ts_manager.cc:194] Registered new tserver with Master: 37e9aca105a7457c8459e7c732a12f20 (127.20.121.131:33721)
I20260929 16:04:57.878638 27137 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:39635
I20260929 16:04:57.889490 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.121.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.189.keytab.
Entry for principal oryx/127.20.121.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.189.keytab.
I20260929 16:04:57.943085 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.189:33763
--webserver_interface=127.20.121.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.189.keytab
--principal=oryx/127.20.121.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:43697,127.20.121.189:33763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:04:58.289146 27600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:58.289532 27600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:58.289686 27600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:58.297566 27600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:58.297750 27600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:58.297856 27600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:58.297959 27600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:58.298061 27600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:58.308072 27600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:43697,127.20.121.189:33763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.189.keytab
--principal=oryx/127.20.121.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.189:33763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.121.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
W20260929 16:04:58.308531 27600 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.121.190:43697,127.20.121.189:33763'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 16:04:58.309711 27600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:58.311089 27600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:58.323107 27605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:58.323101 27606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:58.323315 27608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:58.323993 27600 server_base.cc:1034] running on GCE node
I20260929 16:04:58.324770 27600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:58.326229 27600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:58.327523 27600 hybrid_clock.cc:648] HybridClock initialized: now 1790697898327462 us; error 55 us; skew 500 ppm
I20260929 16:04:58.327916 27600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:58 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697898, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:58.331574 27600 init.cc:377] Logged in from keytab as oryx/127.20.121.189@KRBTEST.COM (short username oryx)
I20260929 16:04:58.333457 27600 webserver.cc:460] Webserver started at http://127.20.121.189:42375/ using document root <none> and password file <none>
I20260929 16:04:58.334334 27600 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:58.334483 27600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:58.334828 27600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:58.338176 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data/instance:
uuid: "0c561530e7bf44299e555f50c33f5317"
format_stamp: "Formatted at 2026-09-29 16:04:58 on dist-test-slave-x07d"
server_key: "917f1b81311b4f8daf0aab995516c662"
server_key_iv: "fff71d04a221aab6f46ebae7698b2f4f"
server_key_version: "encryptionkey@0"
I20260929 16:04:58.339073 27600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal/instance:
uuid: "0c561530e7bf44299e555f50c33f5317"
format_stamp: "Formatted at 2026-09-29 16:04:58 on dist-test-slave-x07d"
server_key: "917f1b81311b4f8daf0aab995516c662"
server_key_iv: "fff71d04a221aab6f46ebae7698b2f4f"
server_key_version: "encryptionkey@0"
I20260929 16:04:58.345053 27600 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260929 16:04:58.349155 27615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:58.349747 27460 heartbeater.cc:503] Master 127.20.121.190:43697 was elected leader, sending a full tablet report...
I20260929 16:04:58.350986 27600 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:04:58.351169 27600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal
uuid: "0c561530e7bf44299e555f50c33f5317"
format_stamp: "Formatted at 2026-09-29 16:04:58 on dist-test-slave-x07d"
server_key: "917f1b81311b4f8daf0aab995516c662"
server_key_iv: "fff71d04a221aab6f46ebae7698b2f4f"
server_key_version: "encryptionkey@0"
I20260929 16:04:58.351379 27600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:58.372189 27600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:58.376688 27600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:58.376968 27600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:58.396018 27600 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.189:33763
I20260929 16:04:58.396029 27666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.189:33763 every 8 connection(s)
I20260929 16:04:58.397445 27600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
I20260929 16:04:58.401693 27667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:58.405493 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 27600
I20260929 16:04:58.410331 27667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:04:58 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.189@KRBTEST.COM
Sep 29 16:04:58 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697898, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.189@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:04:58.432945 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.413970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55996 (local address 127.20.121.190:43697)
0929 16:04:58.414526 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:58.414532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:58.415065 (+ 533us) server_negotiation.cc:407] Connection header received
0929 16:04:58.415291 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:58.415295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:58.415366 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:58.415523 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:58.416982 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.418125 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.420065 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.420472 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.420572 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.425603 (+ 5031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:58.425608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:58.425612 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:58.425651 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:58.428894 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.430044 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.430053 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.430056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.430213 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.430818 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.430823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.430825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.431244 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:58.431460 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:58.432376 (+ 916us) server_negotiation.cc:299] Negotiation successful
0929 16:04:58.432562 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":247,"threads_started":1}
I20260929 16:04:58.434203 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.413968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:55996)
0929 16:04:58.414899 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:58.414945 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:58.415127 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:58.415628 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:58.415636 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:58.416471 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:58.416735 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.416744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.418256 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.418261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.419885 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.419896 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.420743 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.420752 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.420955 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.421702 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:58.421741 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:58.425327 (+ 3586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:58.429099 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.429120 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.429131 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.429769 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.430392 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.430399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.430404 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.430664 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.431497 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:58.431506 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:58.431787 (+ 281us) client_negotiation.cc:769] Sending connection context
0929 16:04:58.432113 (+ 326us) client_negotiation.cc:240] Negotiation successful
0929 16:04:58.432628 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":140,"spinlock_wait_cycles":25984,"thread_start_us":503,"threads_started":1}
I20260929 16:04:58.434223 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.408090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.20.121.189:33763)
0929 16:04:58.408973 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:58.408992 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:58.409048 (+ 56us) server_negotiation.cc:407] Connection header received
0929 16:04:58.411181 (+ 2133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:58.411217 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:58.411853 (+ 636us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:58.412319 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:58.413762 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.415991 (+ 2229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.417932 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.419212 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.419413 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.425265 (+ 5852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:58.425279 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:58.425300 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:58.425337 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:58.428537 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.429530 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.429541 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.429544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.429756 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.430341 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.430348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.430351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.431215 (+ 864us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:58.431590 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:58.432019 (+ 429us) server_negotiation.cc:299] Negotiation successful
0929 16:04:58.432521 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":392,"threads_started":1}
I20260929 16:04:58.436193 27667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:04:58 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697898, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.189@KRBTEST.COM for oryx/127.20.121.189@KRBTEST.COM
I20260929 16:04:58.449707 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.436863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.0.0.1:54820)
0929 16:04:58.437067 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:58.437085 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:58.437174 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:58.437613 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:58.437617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:58.437995 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:58.438212 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.438221 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.439232 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.439236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.440233 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.440246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.441110 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.441116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.441289 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.441992 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:58.442081 (+ 89us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:58.444223 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:58.447296 (+ 3073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.447300 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.447303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.447713 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.448168 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.448172 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.448174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.448303 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.448944 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:58.448948 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:58.449077 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 16:04:58.449199 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 16:04:58.449442 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":1}
I20260929 16:04:58.450055 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.437143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54820 (local address 127.20.121.189:33763)
0929 16:04:58.437311 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:58.437323 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:58.437335 (+ 12us) server_negotiation.cc:407] Connection header received
0929 16:04:58.437368 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:58.437371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:58.437450 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:58.437550 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:58.438336 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.439067 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.440410 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.440804 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.441035 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.444430 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:58.444435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:58.444438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:58.444471 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:58.447142 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.447849 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.447853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.447855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.447999 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.448410 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.448413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.448416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.448739 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:58.448858 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:58.449311 (+ 453us) server_negotiation.cc:299] Negotiation successful
0929 16:04:58.449618 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":60}
I20260929 16:04:58.455024 27667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: Bootstrap starting.
I20260929 16:04:58.458467 27667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:58.459744 27667 log.cc:824] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:58.462898 27667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: No bootstrap required, opened a new log
I20260929 16:04:58.470114 27667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } }
I20260929 16:04:58.470463 27667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:58.470582 27667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c561530e7bf44299e555f50c33f5317, State: Initialized, Role: FOLLOWER
I20260929 16:04:58.471200 27667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } }
I20260929 16:04:58.472173 27674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } } }
I20260929 16:04:58.472556 27674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:04:58.473183 27667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 16:04:58.483378 27685 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 16:04:58.483562 27685 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 16:04:58.486223 27600 master_runner.cc:428] Detected that this master 0c561530e7bf44299e555f50c33f5317 is joining an existing cluster
I20260929 16:04:58.486338 27600 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.121.189:33763
I20260929 16:04:58.512768 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.491269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56000 (local address 127.20.121.190:43697)
0929 16:04:58.491491 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:58.491498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:58.495143 (+ 3645us) server_negotiation.cc:407] Connection header received
0929 16:04:58.495746 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:58.495756 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:58.495922 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:58.496040 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:58.498323 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.499480 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.501585 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.502239 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.502416 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.505083 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:58.505092 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:58.505098 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:58.505150 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:58.509026 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.509992 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.509998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.510002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.510208 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.510768 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.510776 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.510781 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.511217 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:58.511355 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:58.512086 (+ 731us) server_negotiation.cc:299] Negotiation successful
0929 16:04:58.512362 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 16:04:58.514550 27691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.491377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56000)
0929 16:04:58.494970 (+ 3593us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:58.494997 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:58.495485 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:58.496369 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:58.496378 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:58.497403 (+ 1025us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:58.498054 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.498078 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.499739 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.499751 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.501405 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.501419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.502513 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.502520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.502738 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.503567 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:58.503595 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:58.504789 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:58.509225 (+ 4436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.509231 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.509236 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.509797 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.510389 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.510394 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.510399 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.510600 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.511482 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:58.511487 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:58.511734 (+ 247us) client_negotiation.cc:769] Sending connection context
0929 16:04:58.513415 (+ 1681us) client_negotiation.cc:240] Negotiation successful
0929 16:04:58.513838 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3222,"mutex_wait_us":5,"thread_start_us":2991,"threads_started":1}
I20260929 16:04:58.515967 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.495137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54830 (local address 127.20.121.189:33763)
0929 16:04:58.495357 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:58.495363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:58.498744 (+ 3381us) server_negotiation.cc:407] Connection header received
0929 16:04:58.498896 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:58.498901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:58.498986 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:58.499090 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:58.500731 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.502044 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.503951 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.504501 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.504670 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.507327 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:58.507333 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:58.507338 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:58.507372 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:58.511533 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.512625 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.512634 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.512639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.512814 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.513323 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.513331 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.513336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.513814 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:58.514139 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:58.515366 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0929 16:04:58.515625 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 16:04:58.516618 27692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.495321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.0.0.1:54830)
0929 16:04:58.498588 (+ 3267us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:58.498613 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:58.498767 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:58.499260 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:58.499265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:58.500128 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:58.500531 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.500545 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.502261 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.502266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.503761 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.503776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.504724 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.504730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.504926 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.505661 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:58.505690 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:58.506836 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:58.511747 (+ 4911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.511753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.511757 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.512444 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.512960 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.512968 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.512973 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.513165 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.514201 (+ 1036us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:58.514208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:58.514379 (+ 171us) client_negotiation.cc:769] Sending connection context
0929 16:04:58.514575 (+ 196us) client_negotiation.cc:240] Negotiation successful
0929 16:04:58.514922 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3096,"thread_start_us":2862,"threads_started":1}
WARNING: no policy specified for oryx/127.20.121.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.188@KRBTEST.COM" created.
Sep 29 16:04:58 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697896, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.190@KRBTEST.COM for oryx/127.20.121.189@KRBTEST.COM
I20260929 16:04:58.541698 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.523008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54832 (local address 127.20.121.189:33763)
0929 16:04:58.523222 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:58.523231 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:58.523889 (+ 658us) server_negotiation.cc:407] Connection header received
0929 16:04:58.524278 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:58.524284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:58.524366 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:58.524486 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:58.525651 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.526525 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.527966 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.528503 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.528649 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.533359 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:58.533372 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:58.533377 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:58.533425 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:58.537662 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.538900 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.538907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.538911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.539123 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.539690 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.539697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.539700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.540084 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:58.540223 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:58.541013 (+ 790us) server_negotiation.cc:299] Negotiation successful
0929 16:04:58.541277 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 16:04:58.542862 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.522975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.0.0.1:54832)
0929 16:04:58.523758 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:58.523795 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:58.524051 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:58.524535 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:58.524543 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:58.525253 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:58.525469 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.525479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.526680 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.526686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.527811 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.527825 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.528692 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.528699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.528893 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.529648 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:58.529667 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:58.533057 (+ 3390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:58.537919 (+ 4862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.537939 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.537965 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.538713 (+ 748us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.539321 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.539326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.539332 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.539517 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.540355 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:58.540364 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:58.540590 (+ 226us) client_negotiation.cc:769] Sending connection context
0929 16:04:58.541920 (+ 1330us) client_negotiation.cc:240] Negotiation successful
0929 16:04:58.542384 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":390,"threads_started":1}
I20260929 16:04:58.545187 27137 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.121.189:33763: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0c561530e7bf44299e555f50c33f5317"
member_type: NON_VOTER
last_known_addr {
host: "127.20.121.189"
port: 33763
}
attrs {
promote: true
}
}
dest_uuid: "7adf02d0479447608d50d296296b2eaa"
cas_config_opid_index: -1
I20260929 16:04:58.546310 27137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } }
I20260929 16:04:58.549678 27697 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0c561530e7bf44299e555f50c33f5317 (127.20.121.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } } }
I20260929 16:04:58.550794 27642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.188.keytab.
Entry for principal oryx/127.20.121.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.188.keytab.
I20260929 16:04:58.554324 27642 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Refusing update from remote peer 7adf02d0479447608d50d296296b2eaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 16:04:58.554728 27697 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.121.189:33763
I20260929 16:04:58.555670 27600 master.cc:537] Master@127.20.121.189:33763 shutting down...
I20260929 16:04:58.555719 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.188:41459
--webserver_interface=127.20.121.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.188.keytab
--principal=oryx/127.20.121.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:43697,127.20.121.189:33763,127.20.121.188:41459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:04:58.555708 27697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } } }
I20260929 16:04:58.555982 27697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:58.565362 27698 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:04:58.571846 27600 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:58.572330 27600 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:58.572535 27600 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: stopping tablet replica
W20260929 16:04:58.576121 27121 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260929 16:04:58.578163 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.577589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.0.0.1:54842)
0929 16:04:58.577793 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:58.577916 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260929 16:04:58.578397 27121 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa -> Peer 0c561530e7bf44299e555f50c33f5317 (127.20.121.189:33763): Couldn't send request to peer 0c561530e7bf44299e555f50c33f5317. Status: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33763: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 16:04:58.591007 27600 master.cc:552] Master@127.20.121.189:33763 shutdown complete.
I20260929 16:04:58.591161 27600 master_runner.cc:305] Clearing existing system tablet
I20260929 16:04:58.591688 27600 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:58.591812 27600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:58.595538 27600 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260929 16:04:58.597369 27704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:58.597770 27600 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:58.598079 27600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 16:04:58.598167 27600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal
uuid: "0c561530e7bf44299e555f50c33f5317"
format_stamp: "Formatted at 2026-09-29 16:04:58 on dist-test-slave-x07d"
server_key: "917f1b81311b4f8daf0aab995516c662"
server_key_iv: "fff71d04a221aab6f46ebae7698b2f4f"
server_key_version: "encryptionkey@0"
I20260929 16:04:58.603657 27600 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:58.611181 27600 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 16:04:58.611392 27600 log.cc:1196] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 16:04:58.611860 27600 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: Deleting consensus metadata
I20260929 16:04:58.612149 27600 master_runner.cc:315] Copying system tablet from 127.20.121.190:43697
I20260929 16:04:58.614791 27600 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: tablet copy: Beginning tablet copy session from remote peer at address 127.20.121.190:43697
I20260929 16:04:58.635160 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.615627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56016)
0929 16:04:58.618489 (+ 2862us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:58.618512 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:58.618706 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:58.619141 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:58.619146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:58.619717 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:58.620037 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.620051 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.621686 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.621692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.623324 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:58.623343 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.624594 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.624601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:58.624803 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.625640 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:58.625664 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:58.626951 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:58.631500 (+ 4549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.631508 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.631514 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.632220 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.632744 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:58.632752 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:58.632757 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:58.632990 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:58.633830 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:58.633837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:58.634087 (+ 250us) client_negotiation.cc:769] Sending connection context
0929 16:04:58.634262 (+ 175us) client_negotiation.cc:240] Negotiation successful
0929 16:04:58.634618 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2705,"thread_start_us":2584,"threads_started":1}
I20260929 16:04:58.635196 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:58.616166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.20.121.190:43697)
0929 16:04:58.616359 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:58.616366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:58.618655 (+ 2289us) server_negotiation.cc:407] Connection header received
0929 16:04:58.618862 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:58.618869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:58.618966 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:58.619091 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:58.620237 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.621501 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.623560 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:58.624362 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:58.624527 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:58.627221 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:58.627229 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:58.627235 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:58.627283 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:58.631276 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.632404 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.632412 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.632416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.632594 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:58.633168 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:58.633177 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:58.633182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:58.633668 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:58.633865 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:58.634427 (+ 562us) server_negotiation.cc:299] Negotiation successful
0929 16:04:58.634729 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 16:04:58.636600 27157 tablet_copy_service.cc:140] P 7adf02d0479447608d50d296296b2eaa: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0c561530e7bf44299e555f50c33f5317 ({username='oryx', principal='oryx/127.20.121.189@KRBTEST.COM'} at 127.0.0.1:56016)
I20260929 16:04:58.636896 27157 tablet_copy_service.cc:161] P 7adf02d0479447608d50d296296b2eaa: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0c561530e7bf44299e555f50c33f5317 at {username='oryx', principal='oryx/127.20.121.189@KRBTEST.COM'} at 127.0.0.1:56016: session id = 0c561530e7bf44299e555f50c33f5317-00000000000000000000000000000000
I20260929 16:04:58.642273 27157 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 16:04:58.644981 27600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 16:04:58.652153 27600 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: tablet copy: Starting download of 0 data blocks...
I20260929 16:04:58.652529 27600 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: tablet copy: Starting download of 1 WAL segments...
I20260929 16:04:58.655928 27600 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 16:04:58.661763 27157 tablet_copy_service.cc:342] P 7adf02d0479447608d50d296296b2eaa: Request end of tablet copy session 0c561530e7bf44299e555f50c33f5317-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.121.189@KRBTEST.COM'} at 127.0.0.1:56016
I20260929 16:04:58.662073 27157 tablet_copy_service.cc:433] P 7adf02d0479447608d50d296296b2eaa: ending tablet copy session 0c561530e7bf44299e555f50c33f5317-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0c561530e7bf44299e555f50c33f5317
I20260929 16:04:58.677330 27600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:58.681016 27711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:58.681063 27712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:58.690068 27714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:58.697556 27600 server_base.cc:1034] running on GCE node
I20260929 16:04:58.698626 27600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:58.700214 27600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:58.701867 27600 hybrid_clock.cc:648] HybridClock initialized: now 1790697898701796 us; error 42 us; skew 500 ppm
I20260929 16:04:58.702185 27600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:58 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697898, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:58.704793 27600 init.cc:377] Logged in from keytab as oryx/127.20.121.189@KRBTEST.COM (short username oryx)
I20260929 16:04:58.706312 27600 webserver.cc:460] Webserver started at http://127.20.121.189:38931/ using document root <none> and password file <none>
I20260929 16:04:58.706728 27600 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:58.706815 27600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:58.710054 27600 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260929 16:04:58.712116 27721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:58.712597 27600 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 16:04:58.712798 27600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal
uuid: "0c561530e7bf44299e555f50c33f5317"
format_stamp: "Formatted at 2026-09-29 16:04:58 on dist-test-slave-x07d"
server_key: "917f1b81311b4f8daf0aab995516c662"
server_key_iv: "fff71d04a221aab6f46ebae7698b2f4f"
server_key_version: "encryptionkey@0"
I20260929 16:04:58.713032 27600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:58.782884 27600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:58.788065 27600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:58.834172 27600 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.189:33763
I20260929 16:04:58.834201 27772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.189:33763 every 8 connection(s)
I20260929 16:04:58.835438 27600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
I20260929 16:04:58.840974 27773 sys_catalog.cc:263] Verifying existing consensus state
I20260929 16:04:58.845408 27773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: Bootstrap starting.
I20260929 16:04:58.869814 27773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 16:04:58.870492 27773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: Bootstrap complete.
I20260929 16:04:58.871599 27773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } }
I20260929 16:04:58.871865 27773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0c561530e7bf44299e555f50c33f5317, State: Initialized, Role: LEARNER
I20260929 16:04:58.872141 27773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } }
I20260929 16:04:58.873023 27776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } } }
I20260929 16:04:58.873327 27773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:58.873605 27776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: This master's current role is: LEARNER
I20260929 16:04:58.881311 27594 heartbeater.cc:503] Master 127.20.121.190:43697 was elected leader, sending a full tablet report...
I20260929 16:04:58.883469 27787 catalog_manager.cc:1259] Loaded cluster ID: 635f15c821ad4c669fd2eadd14d2dfb2
I20260929 16:04:58.883646 27787 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: loading cluster ID for follower catalog manager: success
I20260929 16:04:58.887827 27787 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: acquiring CA information for follower catalog manager: success
I20260929 16:04:58.890942 27787 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 16:04:59.020202 27701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:59.020519 27701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:59.020669 27701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:59.028666 27701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:04:59.028790 27701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:59.028891 27701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:59.029006 27701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:04:59.029083 27701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:04:59.040087 27701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:43697,127.20.121.189:33763,127.20.121.188:41459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.188.keytab
--principal=oryx/127.20.121.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.188:41459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.121.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:59.041649 27701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:59.043190 27701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:59.056238 27792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:59.056277 27793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:59.056313 27795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:59.057447 27701 server_base.cc:1034] running on GCE node
I20260929 16:04:59.058379 27701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:59.060148 27701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:59.061417 27701 hybrid_clock.cc:648] HybridClock initialized: now 1790697899061337 us; error 114 us; skew 500 ppm
I20260929 16:04:59.061810 27701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:59 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697899, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:59.065682 27701 init.cc:377] Logged in from keytab as oryx/127.20.121.188@KRBTEST.COM (short username oryx)
I20260929 16:04:59.067657 27701 webserver.cc:460] Webserver started at http://127.20.121.188:41885/ using document root <none> and password file <none>
I20260929 16:04:59.068617 27701 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:59.068771 27701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:59.069146 27701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:04:59.072451 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.064653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54846 (local address 127.20.121.189:33763)
0929 16:04:59.065751 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.065755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.065775 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:04:59.065869 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.065876 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.066004 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.066188 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:59.066822 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.068029 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.070527 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.071405 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.071575 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.071707 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.072050 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.072168 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986,"thread_start_us":839,"threads_started":1}
I20260929 16:04:59.072459 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.064304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.0.0.1:54846)
0929 16:04:59.064488 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:59.064515 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:59.064653 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:59.066374 (+ 1721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:59.066382 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:59.066431 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:59.066635 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.066650 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.068205 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.068211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.070267 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.070288 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.071624 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.071631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.071829 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.071896 (+ 67us) client_negotiation.cc:769] Sending connection context
0929 16:04:59.072011 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 16:04:59.072178 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 16:04:59.073410 27701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data/instance:
uuid: "30c111a79abb4f979bd085b0ac6c3579"
format_stamp: "Formatted at 2026-09-29 16:04:59 on dist-test-slave-x07d"
server_key: "a3d800e3596f672b232c438eb534202f"
server_key_iv: "1b2cb66f789bb58598bf7996775fa729"
server_key_version: "encryptionkey@0"
I20260929 16:04:59.073751 27748 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260929 16:04:59.074399 27776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7adf02d0479447608d50d296296b2eaa. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } } }
I20260929 16:04:59.074746 27776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: This master's current role is: LEARNER
I20260929 16:04:59.074710 27701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal/instance:
uuid: "30c111a79abb4f979bd085b0ac6c3579"
format_stamp: "Formatted at 2026-09-29 16:04:59 on dist-test-slave-x07d"
server_key: "a3d800e3596f672b232c438eb534202f"
server_key_iv: "1b2cb66f789bb58598bf7996775fa729"
server_key_version: "encryptionkey@0"
I20260929 16:04:59.080850 27701 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260929 16:04:59.085420 27803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:59.087168 27701 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260929 16:04:59.087481 27701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal
uuid: "30c111a79abb4f979bd085b0ac6c3579"
format_stamp: "Formatted at 2026-09-29 16:04:59 on dist-test-slave-x07d"
server_key: "a3d800e3596f672b232c438eb534202f"
server_key_iv: "1b2cb66f789bb58598bf7996775fa729"
server_key_version: "encryptionkey@0"
I20260929 16:04:59.087742 27701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:59.128404 27701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:59.133332 27701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:04:59.133663 27701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:59.155833 27701 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.188:41459
I20260929 16:04:59.155830 27854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.188:41459 every 8 connection(s)
I20260929 16:04:59.157677 27701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
I20260929 16:04:59.158452 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 27701
I20260929 16:04:59.165086 27855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:04:59.175901 27855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:04:59 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.188@KRBTEST.COM
Sep 29 16:04:59 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697899, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.188@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:04:59.193127 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.167019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39760 (local address 127.20.121.188:41459)
0929 16:04:59.168110 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.168135 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.170751 (+ 2616us) server_negotiation.cc:407] Connection header received
0929 16:04:59.172570 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.172603 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.173238 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.173775 (+ 537us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:59.175607 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.177600 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.179073 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.180006 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.180230 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.185105 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:59.185113 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:59.185126 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:59.185173 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:59.188482 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.189198 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.189204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.189211 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.189356 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.189746 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.189758 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.189760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.190364 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:59.190682 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.191101 (+ 419us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.191565 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":432,"threads_started":1}
I20260929 16:04:59.195097 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.179125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.20.121.190:43697)
0929 16:04:59.179693 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.179699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.180423 (+ 724us) server_negotiation.cc:407] Connection header received
0929 16:04:59.180492 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.180496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.180554 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.180671 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:59.181731 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.182512 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.184186 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.184701 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.184844 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.188572 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:59.188578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:59.188582 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:59.188615 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:59.191648 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.192621 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.192627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.192630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.192784 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.193266 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.193272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.193277 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.193743 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:59.193915 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.194479 (+ 564us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.194800 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":260,"threads_started":1}
I20260929 16:04:59.195201 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.178859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56032)
0929 16:04:59.180132 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:59.180171 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:59.180318 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:59.180731 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:59.180737 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:59.181340 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:59.181577 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.181585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.182664 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.182669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.183974 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.183990 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.184926 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.184935 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.185128 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.185813 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:59.185830 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:59.188351 (+ 2521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:59.191839 (+ 3488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.191848 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.191855 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.192442 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.192898 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.192903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.192906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.193118 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.193875 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:59.193881 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:59.194140 (+ 259us) client_negotiation.cc:769] Sending connection context
0929 16:04:59.194322 (+ 182us) client_negotiation.cc:240] Negotiation successful
0929 16:04:59.194737 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"thread_start_us":849,"threads_started":1}
I20260929 16:04:59.196623 27855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:04:59 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697899, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.188@KRBTEST.COM for oryx/127.20.121.189@KRBTEST.COM
I20260929 16:04:59.210238 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.197326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.0.0.1:54860)
0929 16:04:59.197624 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:59.197644 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:59.197763 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:59.198151 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:59.198155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:59.198884 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:59.199033 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.199041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.200009 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.200013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.200931 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.200942 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.201673 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.201677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.201806 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.202472 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:59.202491 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:59.204542 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:59.207903 (+ 3361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.207907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.207910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.208344 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.208726 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.208729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.208731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.208858 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.209340 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:59.209344 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:59.209467 (+ 123us) client_negotiation.cc:769] Sending connection context
0929 16:04:59.209592 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 16:04:59.209822 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260929 16:04:59.210467 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.197569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54860 (local address 127.20.121.189:33763)
0929 16:04:59.197736 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.197741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.197757 (+ 16us) server_negotiation.cc:407] Connection header received
0929 16:04:59.197876 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.197880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.197939 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.198089 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:59.199155 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.199853 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.201067 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.201514 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.201625 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.204729 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:59.204734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:59.204737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:59.204770 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:59.207750 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.208469 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.208474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.208476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.208618 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.208950 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.208953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.208955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.209229 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:59.209335 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.209746 (+ 411us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.209970 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 16:04:59.211508 27855 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:04:59 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697899, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.188@KRBTEST.COM for oryx/127.20.121.188@KRBTEST.COM
I20260929 16:04:59.226497 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.212563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:41459 (local address 127.0.0.1:39772)
0929 16:04:59.212753 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:59.212774 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:59.212916 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:59.213236 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:59.213240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:59.213631 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:59.213763 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.213771 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.214809 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.214816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.215673 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.215684 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.216448 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.216452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.216747 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.217386 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:59.217403 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:59.219651 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:59.223492 (+ 3841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.223497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.223502 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.223910 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.224347 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.224350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.224352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.224502 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.225115 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:59.225119 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:59.225322 (+ 203us) client_negotiation.cc:769] Sending connection context
0929 16:04:59.225527 (+ 205us) client_negotiation.cc:240] Negotiation successful
0929 16:04:59.225824 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":20}
I20260929 16:04:59.226619 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.212563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39772 (local address 127.20.121.188:41459)
0929 16:04:59.212753 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.212761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.212851 (+ 90us) server_negotiation.cc:407] Connection header received
0929 16:04:59.213071 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.213074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.213123 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.213225 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:59.213893 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.214683 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.215806 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.216308 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.216438 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.219843 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:59.219848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:59.219851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:59.219880 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:59.222872 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.224059 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.224064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.224066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.224236 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.224645 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.224649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.224652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.224956 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:59.225095 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.225660 (+ 565us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.225962 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":138,"server-negotiator.queue_time_us":69}
I20260929 16:04:59.231913 27855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Bootstrap starting.
I20260929 16:04:59.235603 27855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Neither blocks nor log segments found. Creating new log.
I20260929 16:04:59.236866 27855 log.cc:824] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Log is configured to *not* fsync() on all Append() calls
I20260929 16:04:59.240348 27855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: No bootstrap required, opened a new log
I20260929 16:04:59.249073 27855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } }
I20260929 16:04:59.249455 27855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:04:59.249608 27855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c111a79abb4f979bd085b0ac6c3579, State: Initialized, Role: FOLLOWER
I20260929 16:04:59.250272 27855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } }
I20260929 16:04:59.251181 27862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } } }
I20260929 16:04:59.251566 27862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:04:59.252249 27855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 16:04:59.262475 27873 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 16:04:59.262646 27873 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 16:04:59.268038 27701 master_runner.cc:428] Detected that this master 30c111a79abb4f979bd085b0ac6c3579 is joining an existing cluster
I20260929 16:04:59.268163 27701 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.121.188:41459
I20260929 16:04:59.292725 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.273301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.20.121.190:43697)
0929 16:04:59.273487 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.273493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.273609 (+ 116us) server_negotiation.cc:407] Connection header received
0929 16:04:59.273879 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.273885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.274040 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.274126 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:59.275926 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.277414 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.280436 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.281341 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.281467 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.285038 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:59.285047 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:59.285052 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:59.285094 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:59.289040 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.289863 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.289868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.289872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.290075 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.290556 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.290561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.290564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.290918 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:59.291381 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.292093 (+ 712us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.292350 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 16:04:59.294029 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.272819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56040)
0929 16:04:59.273504 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:59.273525 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:59.273721 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:59.274361 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:59.274365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:59.275212 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:59.275700 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.275719 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.277816 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.277824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.280200 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.280221 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.281815 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.281824 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.282243 (+ 419us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.283105 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:59.283134 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:59.284804 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:59.289229 (+ 4425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.289234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.289238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.289717 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.290227 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.290232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.290235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.290418 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.291478 (+ 1060us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:59.291482 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:59.291755 (+ 273us) client_negotiation.cc:769] Sending connection context
0929 16:04:59.292677 (+ 922us) client_negotiation.cc:240] Negotiation successful
0929 16:04:59.293460 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":666,"threads_started":1}
I20260929 16:04:59.297191 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.272830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39774 (local address 127.20.121.188:41459)
0929 16:04:59.273054 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.273063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.279885 (+ 6822us) server_negotiation.cc:407] Connection header received
0929 16:04:59.280334 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.280340 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.280402 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.280463 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:59.282338 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.283491 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.285718 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.286448 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.286536 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.289454 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:59.289459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:59.289464 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:59.289498 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:59.293776 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.294805 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.294809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.294812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.294987 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.295351 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.295355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.295359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.295779 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:59.295979 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.296495 (+ 516us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.296737 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 16:04:59.299114 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.272983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:41459 (local address 127.0.0.1:39774)
0929 16:04:59.279787 (+ 6804us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:59.279808 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:59.280200 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:59.280657 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:59.280662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:59.281802 (+ 1140us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:59.282180 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.282193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.283716 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.283721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.285575 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.285590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.286824 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.286830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.287031 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.287707 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:59.287735 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:59.289288 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:59.294049 (+ 4761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.294054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.294059 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.294637 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.295121 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.295125 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.295129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.295253 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.296052 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:59.296056 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:59.296245 (+ 189us) client_negotiation.cc:769] Sending connection context
0929 16:04:59.297429 (+ 1184us) client_negotiation.cc:240] Negotiation successful
0929 16:04:59.298429 (+ 1000us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6647,"mutex_wait_us":82,"thread_start_us":6909,"threads_started":1}
I20260929 16:04:59.302043 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 27198
I20260929 16:04:59.302235 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.274357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54872 (local address 127.20.121.189:33763)
0929 16:04:59.274557 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.274563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.283463 (+ 8900us) server_negotiation.cc:407] Connection header received
0929 16:04:59.283931 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.283938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.284024 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.284094 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:59.285724 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.286871 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.289396 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.290083 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.290182 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.294843 (+ 4661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:59.294853 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:59.294858 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:59.294918 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:59.298379 (+ 3461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.299855 (+ 1476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.299864 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.299870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.300040 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.300494 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.300499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.300503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.300854 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:59.300990 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.301622 (+ 632us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.301855 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 16:04:59.302399 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.273917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.0.0.1:54872)
0929 16:04:59.283342 (+ 9425us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:59.283361 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:59.283713 (+ 352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:59.284332 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:59.284336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:59.285094 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:59.285530 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.285542 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.287065 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.287071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.289236 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.289245 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.291399 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.291409 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.291618 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.292310 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:59.292331 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:59.294640 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:59.298876 (+ 4236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.298884 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.298891 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.299698 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.300182 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.300187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.300191 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.300351 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.301084 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:59.301089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:59.301321 (+ 232us) client_negotiation.cc:769] Sending connection context
0929 16:04:59.301482 (+ 161us) client_negotiation.cc:240] Negotiation successful
0929 16:04:59.301796 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9271,"mutex_wait_us":1,"thread_start_us":9585,"threads_started":1}
Sep 29 16:04:59 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697896, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.190@KRBTEST.COM for oryx/127.20.121.188@KRBTEST.COM
I20260929 16:04:59.321143 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.303078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39782 (local address 127.20.121.188:41459)
0929 16:04:59.303311 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.303317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.303333 (+ 16us) server_negotiation.cc:407] Connection header received
0929 16:04:59.303413 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.303417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.303506 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.303615 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:59.304712 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.305515 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.307892 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.308659 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.308795 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.313376 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:59.313384 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:59.313388 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:59.313428 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:59.317198 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.318268 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.318277 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.318284 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.318430 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.318962 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.318968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.318973 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.319342 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:59.319904 (+ 562us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.320541 (+ 637us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.320778 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260929 16:04:59.321646 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.300590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:41459 (local address 127.0.0.1:39782)
0929 16:04:59.300795 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:59.300814 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:59.300915 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:59.303658 (+ 2743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:59.303663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:59.304287 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:59.304552 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.304569 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.305655 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.305663 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.307691 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.307714 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.308863 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.308872 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.309120 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.309857 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:59.309876 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:59.313157 (+ 3281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:59.317519 (+ 4362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.317527 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.317533 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.318116 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.318624 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.318629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.318634 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.318825 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.319992 (+ 1167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:59.320000 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:59.320222 (+ 222us) client_negotiation.cc:769] Sending connection context
0929 16:04:59.320387 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 16:04:59.320730 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
Sep 29 16:04:59 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.129@KRBTEST.COM
I20260929 16:04:59.323338 27137 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.121.188:41459: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579"
member_type: NON_VOTER
last_known_addr {
host: "127.20.121.188"
port: 41459
}
attrs {
promote: true
}
}
dest_uuid: "7adf02d0479447608d50d296296b2eaa"
cas_config_opid_index: 5
I20260929 16:04:59.324415 27137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } }
I20260929 16:04:59.326417 27697 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 30c111a79abb4f979bd085b0ac6c3579 (127.20.121.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:04:59.327080 27748 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 LEARNER]: Refusing update from remote peer 7adf02d0479447608d50d296296b2eaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 16:04:59.331099 27697 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.121.188:41459
I20260929 16:04:59.332064 27701 master.cc:537] Master@127.20.121.188:41459 shutting down...
I20260929 16:04:59.332271 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.303007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.20.121.129:44627)
0929 16:04:59.309453 (+ 6446us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.309466 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.309498 (+ 32us) server_negotiation.cc:407] Connection header received
0929 16:04:59.309567 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.309577 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.309923 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.310134 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:59.311808 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.313546 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.315648 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.316654 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.321674 (+ 5020us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.323469 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:59.323484 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:59.323505 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:59.323559 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:59.327747 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.328725 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.328735 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.328739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.328933 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.329369 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.329375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.329378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.329824 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:59.330150 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.331426 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.331711 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6259,"thread_start_us":6092,"threads_started":1}
I20260929 16:04:59.334874 27321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:04:59.354800 27884 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 16:04:59.360980 27748 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 30c111a79abb4f979bd085b0ac6c3579 (127.20.121.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
W20260929 16:04:59.364168 27122 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa -> Peer 30c111a79abb4f979bd085b0ac6c3579 (127.20.121.188:41459): Couldn't send request to peer 30c111a79abb4f979bd085b0ac6c3579. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 16:04:59.366001 27776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:04:59.366427 27776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: This master's current role is: LEARNER
I20260929 16:04:59.368063 27697 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa: attempting to promote NON_VOTER 0c561530e7bf44299e555f50c33f5317 to VOTER
I20260929 16:04:59.368070 27884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: NON_VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: true } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:04:59.368645 27884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:59.369059 27697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } }
I20260929 16:04:59.371510 27701 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260929 16:04:59.372061 27701 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260929 16:04:59.372273 27701 tablet_replica.cc:330] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: stopping tablet replica
W20260929 16:04:59.374122 27122 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa -> Peer 30c111a79abb4f979bd085b0ac6c3579 (127.20.121.188:41459): Couldn't send request to peer 30c111a79abb4f979bd085b0ac6c3579. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 16:04:59.374573 27748 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 LEARNER]: Refusing update from remote peer 7adf02d0479447608d50d296296b2eaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 16:04:59.375512 27884 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 16:04:59.379060 27701 master.cc:552] Master@127.20.121.188:41459 shutdown complete.
I20260929 16:04:59.379166 27701 master_runner.cc:305] Clearing existing system tablet
I20260929 16:04:59.379577 27701 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:59.379722 27701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:59.381917 27701 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260929 16:04:59.383463 27891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:59.383800 27701 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:59.384142 27701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260929 16:04:59.384248 27701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal
uuid: "30c111a79abb4f979bd085b0ac6c3579"
format_stamp: "Formatted at 2026-09-29 16:04:59 on dist-test-slave-x07d"
server_key: "a3d800e3596f672b232c438eb534202f"
server_key_iv: "1b2cb66f789bb58598bf7996775fa729"
server_key_version: "encryptionkey@0"
I20260929 16:04:59.385346 27883 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0c561530e7bf44299e555f50c33f5317 (127.20.121.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:04:59.386413 27748 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 0c561530e7bf44299e555f50c33f5317 (127.20.121.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:04:59.389123 27701 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:04:59.390409 27776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:04:59.390692 27776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:04:59.391750 27697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:04:59.392041 27697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:59.392338 27883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:04:59.392649 27883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: This master's current role is: LEADER
I20260929 16:04:59.395784 27701 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 16:04:59.395977 27701 log.cc:1196] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 16:04:59.396420 27701 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Deleting consensus metadata
I20260929 16:04:59.396692 27701 master_runner.cc:315] Copying system tablet from 127.20.121.190:43697
I20260929 16:04:59.403764 27701 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: tablet copy: Beginning tablet copy session from remote peer at address 127.20.121.190:43697
I20260929 16:04:59.416766 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.404926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56052 (local address 127.20.121.190:43697)
0929 16:04:59.405155 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.405161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.405234 (+ 73us) server_negotiation.cc:407] Connection header received
0929 16:04:59.405528 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.405534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.405609 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.405718 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:04:59.406600 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.407343 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.408598 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.409047 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.409189 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.411042 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:04:59.411048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:04:59.411052 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:04:59.411092 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:04:59.413985 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.414764 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.414771 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.414775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.414921 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:04:59.415315 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:04:59.415321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:04:59.415325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:04:59.415625 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:04:59.415782 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.416183 (+ 401us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.416389 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260929 16:04:59.416831 27897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.404615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56052)
0929 16:04:59.405126 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:59.405152 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:59.405351 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:59.405746 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:59.405751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:59.406245 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:04:59.406455 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.406463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.407497 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.407502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.408450 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.408461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.409205 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.409211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.409387 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.410098 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:04:59.410117 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:04:59.410841 (+ 724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:04:59.414139 (+ 3298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.414145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.414149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.414640 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.415043 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:04:59.415047 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:04:59.415051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:04:59.415205 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:04:59.415742 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:04:59.415747 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:04:59.415921 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 16:04:59.416070 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 16:04:59.416386 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":550,"threads_started":1}
I20260929 16:04:59.417750 27157 tablet_copy_service.cc:140] P 7adf02d0479447608d50d296296b2eaa: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 30c111a79abb4f979bd085b0ac6c3579 ({username='oryx', principal='oryx/127.20.121.188@KRBTEST.COM'} at 127.0.0.1:56052)
I20260929 16:04:59.417915 27157 tablet_copy_service.cc:161] P 7adf02d0479447608d50d296296b2eaa: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 30c111a79abb4f979bd085b0ac6c3579 at {username='oryx', principal='oryx/127.20.121.188@KRBTEST.COM'} at 127.0.0.1:56052: session id = 30c111a79abb4f979bd085b0ac6c3579-00000000000000000000000000000000
I20260929 16:04:59.420178 27157 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 16:04:59.421861 27701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 16:04:59.427326 27701 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: tablet copy: Starting download of 0 data blocks...
I20260929 16:04:59.427680 27701 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: tablet copy: Starting download of 1 WAL segments...
I20260929 16:04:59.430171 27701 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 16:04:59.433722 27157 tablet_copy_service.cc:342] P 7adf02d0479447608d50d296296b2eaa: Request end of tablet copy session 30c111a79abb4f979bd085b0ac6c3579-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.121.188@KRBTEST.COM'} at 127.0.0.1:56052
I20260929 16:04:59.433877 27157 tablet_copy_service.cc:433] P 7adf02d0479447608d50d296296b2eaa: ending tablet copy session 30c111a79abb4f979bd085b0ac6c3579-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 30c111a79abb4f979bd085b0ac6c3579
I20260929 16:04:59.444731 27701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:59.447734 27898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:59.449358 27899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:59.462311 27901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:59.463687 27701 server_base.cc:1034] running on GCE node
I20260929 16:04:59.464094 27701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:59.464761 27701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:59.466140 27701 hybrid_clock.cc:648] HybridClock initialized: now 1790697899466080 us; error 38 us; skew 500 ppm
I20260929 16:04:59.466390 27701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:59 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697899, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:59.468701 27701 init.cc:377] Logged in from keytab as oryx/127.20.121.188@KRBTEST.COM (short username oryx)
I20260929 16:04:59.470237 27701 webserver.cc:460] Webserver started at http://127.20.121.188:35013/ using document root <none> and password file <none>
I20260929 16:04:59.470587 27701 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:59.470708 27701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:59.473181 27701 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260929 16:04:59.475274 27908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:04:59.475822 27701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 16:04:59.476008 27701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal
uuid: "30c111a79abb4f979bd085b0ac6c3579"
format_stamp: "Formatted at 2026-09-29 16:04:59 on dist-test-slave-x07d"
server_key: "a3d800e3596f672b232c438eb534202f"
server_key_iv: "1b2cb66f789bb58598bf7996775fa729"
server_key_version: "encryptionkey@0"
I20260929 16:04:59.476227 27701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:04:59.508337 27701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:04:59.511943 27701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:04:59.536996 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 27198
I20260929 16:04:59.553603 27701 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.188:41459
I20260929 16:04:59.553675 27959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.188:41459 every 8 connection(s)
I20260929 16:04:59.554867 27701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
I20260929 16:04:59.561131 27960 sys_catalog.cc:263] Verifying existing consensus state
I20260929 16:04:59.564311 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:44627
--local_ip_for_outbound_sockets=127.20.121.129
--tserver_master_addrs=127.20.121.190:43697
--webserver_port=36407
--webserver_interface=127.20.121.129
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab
--principal=oryx/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.121.190:43697,127.20.121.189:33763
--tserver_master_addrs=127.20.121.190:43697,127.20.121.189:33763,127.20.121.188:41459 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:04:59.565768 27960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Bootstrap starting.
I20260929 16:04:59.587354 27960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 16:04:59.588112 27960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Bootstrap complete.
I20260929 16:04:59.589371 27960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } }
I20260929 16:04:59.589666 27960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 30c111a79abb4f979bd085b0ac6c3579, State: Initialized, Role: LEARNER
I20260929 16:04:59.589938 27960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } }
I20260929 16:04:59.590869 27964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:04:59.591298 27964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: This master's current role is: LEARNER
I20260929 16:04:59.591149 27960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:04:59.602506 27975 catalog_manager.cc:1259] Loaded cluster ID: 635f15c821ad4c669fd2eadd14d2dfb2
I20260929 16:04:59.602674 27975 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: loading cluster ID for follower catalog manager: success
I20260929 16:04:59.606175 27975 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: acquiring CA information for follower catalog manager: success
I20260929 16:04:59.608801 27975 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 16:04:59.938232 27962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:04:59.938552 27962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:04:59.938705 27962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:04:59.948470 27962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:04:59.948622 27962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:04:59.948868 27962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:04:59.960690 27962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab
--principal=oryx/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:44627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=36407
--tserver_master_addrs=127.20.121.190:43697,127.20.121.189:33763,127.20.121.188:41459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:04:59.962421 27962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:04:59.963864 27962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:04:59.978498 27981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:59.978590 27982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:04:59.978535 27984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:04:59.983956 27962 server_base.cc:1034] running on GCE node
I20260929 16:04:59.984922 27962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:04:59.986572 27962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:04:59.987869 27962 hybrid_clock.cc:648] HybridClock initialized: now 1790697899987768 us; error 79 us; skew 500 ppm
I20260929 16:04:59.988296 27962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:04:59 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697899, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:04:59.992007 27962 init.cc:377] Logged in from keytab as oryx/127.20.121.129@KRBTEST.COM (short username oryx)
I20260929 16:04:59.993214 27986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.984539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:41459 (local address 127.0.0.1:39788)
0929 16:04:59.985193 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0929 16:04:59.985210 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:04:59.985319 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:04:59.987398 (+ 2079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:04:59.987403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:04:59.987437 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:04:59.987640 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.987651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.989060 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.989066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.990284 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:04:59.990300 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.991820 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.991826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:04:59.992502 (+ 676us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.992553 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 16:04:59.992671 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 16:04:59.992826 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":256,"threads_started":1}
I20260929 16:04:59.993255 27987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:04:59.984547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.20.121.188:41459)
0929 16:04:59.987064 (+ 2517us) server_negotiation.cc:206] Beginning negotiation
0929 16:04:59.987069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:04:59.987090 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:04:59.987153 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:04:59.987156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:04:59.987226 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:04:59.987375 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:04:59.987823 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.988894 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.990460 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:04:59.991623 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:04:59.991808 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:04:59.992029 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 16:04:59.992717 (+ 688us) server_negotiation.cc:299] Negotiation successful
0929 16:04:59.992930 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2394,"thread_start_us":2287,"threads_started":1}
I20260929 16:04:59.994503 27962 webserver.cc:460] Webserver started at http://127.20.121.129:36407/ using document root <none> and password file <none>
I20260929 16:04:59.994666 27935 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260929 16:04:59.995455 27962 fs_manager.cc:359] Metadata directory not provided
I20260929 16:04:59.995621 27962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:04:59.995457 27964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7adf02d0479447608d50d296296b2eaa. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:04:59.995754 27964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: This master's current role is: LEARNER
I20260929 16:05:00.002494 27962 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 16:05:00.005898 27993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:00.007766 27962 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:05:00.008010 27962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0"
format_stamp: "Formatted at 2026-09-29 16:04:56 on dist-test-slave-x07d"
server_key: "9f188ac29c78bbc72acf54e8885d49e7"
server_key_iv: "f6b69e1974d4fd06a5268509647a6082"
server_key_version: "encryptionkey@0"
I20260929 16:05:00.009093 27962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:00.025278 27962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:00.031209 27962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:00.031579 27962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:00.032699 27962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:00.034297 27962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:00.034425 27962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:00.034559 27962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:00.034650 27962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:00.073977 27962 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:44627
I20260929 16:05:00.074317 28105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:44627 every 8 connection(s)
I20260929 16:05:00.075788 27962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:05:00.080255 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 27962
I20260929 16:05:00.080580 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 27332
Sep 29 16:05:00 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697899, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.188@KRBTEST.COM
Sep 29 16:05:00 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697899, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:05:00.113066 27987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.080104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:39459 (local address 127.20.121.188:41459)
0929 16:05:00.080644 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:00.080650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:00.082062 (+ 1412us) server_negotiation.cc:407] Connection header received
0929 16:05:00.086581 (+ 4519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:00.086592 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:00.086727 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:00.086823 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:00.090386 (+ 3563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.091891 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.095663 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.096207 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.096350 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.102737 (+ 6387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:00.102744 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:00.102749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:00.102793 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:00.106674 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.107889 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.107896 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.107899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.108079 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.108720 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.108727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.108731 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.109104 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:00.109267 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:00.112330 (+ 3063us) server_negotiation.cc:299] Negotiation successful
0929 16:05:00.112658 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403}
Sep 29 16:05:00 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.130@KRBTEST.COM
I20260929 16:05:00.115672 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.080832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:41459 (local address 127.20.121.129:39459)
0929 16:05:00.081856 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:00.081916 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:00.085915 (+ 3999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:00.087688 (+ 1773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:00.087707 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:00.088839 (+ 1132us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:00.090038 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.090076 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.092215 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.092225 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.095399 (+ 3174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.095418 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.096735 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.096741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.097092 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.098647 (+ 1555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:00.098688 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:00.102440 (+ 3752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:00.106876 (+ 4436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.106891 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.106917 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.107708 (+ 791us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.108277 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.108283 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.108287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.108575 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.111246 (+ 2671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:00.111255 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:00.111811 (+ 556us) client_negotiation.cc:769] Sending connection context
0929 16:05:00.113325 (+ 1514us) client_negotiation.cc:240] Negotiation successful
0929 16:05:00.114002 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":450,"threads_started":1}
I20260929 16:05:00.117615 28106 heartbeater.cc:347] Connected to a master server at 127.20.121.188:41459
I20260929 16:05:00.117977 28106 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:00.119055 28106 heartbeater.cc:511] Master 127.20.121.188:41459 requested a full tablet report, sending...
I20260929 16:05:00.119779 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.080891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.129:53733)
0929 16:05:00.099499 (+ 18608us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:00.099653 (+ 154us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:00.099897 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:00.100583 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:00.100590 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:00.101148 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:00.101464 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.101477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.103110 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.103116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.105476 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.105495 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.106926 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.106932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.107148 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.107887 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:00.107912 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:00.111059 (+ 3147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:00.115524 (+ 4465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.115529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.115533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.117151 (+ 1618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.117611 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.117616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.117619 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.117774 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.118576 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:00.118580 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:00.118752 (+ 172us) client_negotiation.cc:769] Sending connection context
0929 16:05:00.118931 (+ 179us) client_negotiation.cc:240] Negotiation successful
0929 16:05:00.119282 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18445,"spinlock_wait_cycles":21376,"thread_start_us":18130,"threads_started":1}
I20260929 16:05:00.120812 28110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.080028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:53733 (local address 127.20.121.190:43697)
0929 16:05:00.080639 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:00.080646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:00.099772 (+ 19126us) server_negotiation.cc:407] Connection header received
0929 16:05:00.100057 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:00.100064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:00.100165 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:00.100252 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:00.101685 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.102905 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.105708 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.106458 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.106604 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.111308 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:00.111316 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:00.111321 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:00.111364 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:00.115320 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.117272 (+ 1952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.117278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.117282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.117456 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.117918 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.117925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.117930 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.118362 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:00.118503 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:00.120144 (+ 1641us) server_negotiation.cc:299] Negotiation successful
0929 16:05:00.120402 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":292,"threads_started":1}
I20260929 16:05:00.122747 27925 ts_manager.cc:194] Registered new tserver with Master: 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627)
I20260929 16:05:00.124256 28107 heartbeater.cc:347] Connected to a master server at 127.20.121.190:43697
I20260929 16:05:00.124464 28107 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:00.124977 28107 heartbeater.cc:511] Master 127.20.121.190:43697 requested a full tablet report, sending...
Sep 29 16:05:00 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697899, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.189@KRBTEST.COM
I20260929 16:05:00.125562 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.082654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.20.121.130:37735)
0929 16:05:00.095284 (+ 12630us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:00.095298 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:00.095345 (+ 47us) server_negotiation.cc:407] Connection header received
0929 16:05:00.095461 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:00.095471 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:00.095902 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:00.098655 (+ 2753us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:00.099718 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.101764 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.105168 (+ 3404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.106845 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.107022 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.115589 (+ 8567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:00.115611 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:00.115629 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:00.115675 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:00.120436 (+ 4761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.121482 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.121494 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.121497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.121682 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.122266 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.122274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.122278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.123158 (+ 880us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:00.123478 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:00.124775 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0929 16:05:00.125096 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12182,"thread_start_us":12091,"threads_started":1}
I20260929 16:05:00.126622 27136 ts_manager.cc:194] Re-registered known tserver with Master: 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627)
I20260929 16:05:00.127050 27455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:00.127877 27136 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:53733
I20260929 16:05:00.136549 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.099969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:41383 (local address 127.20.121.189:33763)
0929 16:05:00.114478 (+ 14509us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:00.114487 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:00.114517 (+ 30us) server_negotiation.cc:407] Connection header received
0929 16:05:00.114588 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:00.114599 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:00.115497 (+ 898us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:00.115617 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:00.117230 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.118572 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.120728 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.121434 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.121602 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.127578 (+ 5976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:00.127589 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:00.127596 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:00.127655 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:00.132874 (+ 5219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.134238 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.134248 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.134251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.134399 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.134895 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.134899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.134901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.135261 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:00.135383 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:00.135898 (+ 515us) server_negotiation.cc:299] Negotiation successful
0929 16:05:00.136175 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14330,"thread_start_us":14197,"threads_started":1}
I20260929 16:05:00.136595 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.106082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.20.121.129:41383)
0929 16:05:00.110300 (+ 4218us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:00.110321 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:00.110443 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:00.115898 (+ 5455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:00.115903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:00.116668 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:00.117025 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.117040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.118759 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.118766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.120548 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.120564 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.121902 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.121913 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.122138 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.123156 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:00.123180 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:00.127227 (+ 4047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:00.133181 (+ 5954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.133190 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.133198 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.133990 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.134552 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.134557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.134562 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.134749 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.135407 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:00.135413 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:00.135592 (+ 179us) client_negotiation.cc:769] Sending connection context
0929 16:05:00.135741 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 16:05:00.136149 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4063,"thread_start_us":3409,"threads_started":1}
I20260929 16:05:00.137655 28108 heartbeater.cc:347] Connected to a master server at 127.20.121.189:33763
I20260929 16:05:00.137840 28108 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:00.138361 28108 heartbeater.cc:511] Master 127.20.121.189:33763 requested a full tablet report, sending...
I20260929 16:05:00.140556 27738 ts_manager.cc:194] Registered new tserver with Master: 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627)
I20260929 16:05:00.225009 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 27332
I20260929 16:05:00.242331 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:37735
--local_ip_for_outbound_sockets=127.20.121.130
--tserver_master_addrs=127.20.121.190:43697
--webserver_port=42903
--webserver_interface=127.20.121.130
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab
--principal=oryx/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.121.190:43697,127.20.121.189:33763
--tserver_master_addrs=127.20.121.190:43697,127.20.121.189:33763,127.20.121.188:41459 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:05:00.472496 27976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: NON_VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: true } } }
I20260929 16:05:00.472888 27976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:00.474262 27976 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa: attempting to promote NON_VOTER 30c111a79abb4f979bd085b0ac6c3579 to VOTER
I20260929 16:05:00.475041 27976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } }
I20260929 16:05:00.477761 27748 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Refusing update from remote peer 7adf02d0479447608d50d296296b2eaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260929 16:05:00.477986 27935 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 LEARNER]: Refusing update from remote peer 7adf02d0479447608d50d296296b2eaa: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260929 16:05:00.478667 28119 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260929 16:05:00.479501 27976 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260929 16:05:00.483144 27976 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 30c111a79abb4f979bd085b0ac6c3579 (127.20.121.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:00.483882 27748 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 30c111a79abb4f979bd085b0ac6c3579 (127.20.121.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:00.487498 27935 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 30c111a79abb4f979bd085b0ac6c3579 (127.20.121.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:00.491233 28126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:00.492180 28126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:05:00.493219 27964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:00.493459 27964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:05:00.495236 28119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:00.495589 28119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:00.495436 27976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:00.495715 27976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:00.495208 28120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7adf02d0479447608d50d296296b2eaa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:00.495925 28120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa [sys.catalog]: This master's current role is: LEADER
W20260929 16:05:00.606851 28118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:00.607206 28118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:00.607375 28118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:00.615320 28118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:00.615497 28118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:00.615679 28118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:05:00.626381 28118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab
--principal=oryx/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:37735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=42903
--tserver_master_addrs=127.20.121.190:43697,127.20.121.189:33763,127.20.121.188:41459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:00.627807 28118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:00.629189 28118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:00.644299 28135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:00.644215 28133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:00.644322 28132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:00.645025 28118 server_base.cc:1034] running on GCE node
I20260929 16:05:00.645737 28118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:00.646847 28118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:00.648089 28118 hybrid_clock.cc:648] HybridClock initialized: now 1790697900648032 us; error 42 us; skew 500 ppm
I20260929 16:05:00.648468 28118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:00 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697900, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:00.652151 28118 init.cc:377] Logged in from keytab as oryx/127.20.121.130@KRBTEST.COM (short username oryx)
I20260929 16:05:00.653985 28118 webserver.cc:460] Webserver started at http://127.20.121.130:42903/ using document root <none> and password file <none>
I20260929 16:05:00.654831 28118 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:00.654969 28118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:00.661337 28118 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260929 16:05:00.665115 28142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:00.667091 28118 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:05:00.667328 28118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "5188495bc43c49e48ebdf870aa0b5d80"
format_stamp: "Formatted at 2026-09-29 16:04:57 on dist-test-slave-x07d"
server_key: "0ed91f2cdf57ce62cc0059abec526760"
server_key_iv: "6a83a79e1e31520b2074e500d56ce888"
server_key_version: "encryptionkey@0"
I20260929 16:05:00.668399 28118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:00.698771 28118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:00.703318 28118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:00.703682 28118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:00.704756 28118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:00.706401 28118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:00.706496 28118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:00.706629 28118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:00.706730 28118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:00.740765 28118 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:37735
I20260929 16:05:00.740857 28254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:37735 every 8 connection(s)
I20260929 16:05:00.742298 28118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:05:00.747053 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 28118
I20260929 16:05:00.747372 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 27466
Sep 29 16:05:00 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697900, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
Sep 29 16:05:00 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697900, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.188@KRBTEST.COM
Sep 29 16:05:00 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697900, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.189@KRBTEST.COM
I20260929 16:05:00.776682 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.746057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:54507 (local address 127.20.121.190:43697)
0929 16:05:00.746599 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:00.746608 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:00.755196 (+ 8588us) server_negotiation.cc:407] Connection header received
0929 16:05:00.755360 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:00.755368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:00.755466 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:00.755576 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:00.757330 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.758575 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.760505 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.761143 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.761238 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.766580 (+ 5342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:00.766586 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:00.766590 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:00.766637 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:00.771303 (+ 4666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.772568 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.772576 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.772580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.772752 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.773359 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.773364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.773367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.773730 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:00.773861 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:00.775073 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0929 16:05:00.775361 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":251,"threads_started":1}
I20260929 16:05:00.776695 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.745413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:35451 (local address 127.20.121.188:41459)
0929 16:05:00.746058 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:00.746064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:00.747717 (+ 1653us) server_negotiation.cc:407] Connection header received
0929 16:05:00.751220 (+ 3503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:00.751231 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:00.751366 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:00.751468 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:00.755404 (+ 3936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.757042 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.759724 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.760376 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.760482 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.767082 (+ 6600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:00.767091 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:00.767098 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:00.767150 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:00.772285 (+ 5135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.773610 (+ 1325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.773617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.773622 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.773806 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.774459 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.774465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.774470 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.774855 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:00.775010 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:00.775945 (+ 935us) server_negotiation.cc:299] Negotiation successful
0929 16:05:00.776272 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":260,"threads_started":1}
Sep 29 16:05:00 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.131@KRBTEST.COM
I20260929 16:05:00.778558 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.746025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.130:54507)
0929 16:05:00.755051 (+ 9026us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:00.755069 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:00.755242 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:00.755753 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:00.755758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:00.756829 (+ 1071us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:00.757149 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.757163 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.758779 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.758784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.760342 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.760356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.761513 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.761521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.761736 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.762344 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:00.762365 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:00.766258 (+ 3893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:00.771547 (+ 5289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.771560 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.771583 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.772350 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.772913 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.772919 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.772924 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.773206 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.773983 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:00.773993 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:00.774548 (+ 555us) client_negotiation.cc:769] Sending connection context
0929 16:05:00.777048 (+ 2500us) client_negotiation.cc:240] Negotiation successful
0929 16:05:00.777361 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8853,"mutex_wait_us":41,"spinlock_wait_cycles":37760,"thread_start_us":558,"threads_started":1}
I20260929 16:05:00.778586 28262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.746032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:41459 (local address 127.20.121.130:35451)
0929 16:05:00.747197 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:00.747242 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:00.750777 (+ 3535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:00.752184 (+ 1407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:00.752202 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:00.753261 (+ 1059us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:00.754388 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.754423 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.757285 (+ 2862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.757293 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.759535 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.759549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.760820 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.760827 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.761119 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.762126 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:00.762162 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:00.766809 (+ 4647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:00.772741 (+ 5932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.772745 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.772747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.773433 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.773952 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.773961 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.773968 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.774218 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.775084 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:00.775090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:00.775271 (+ 181us) client_negotiation.cc:769] Sending connection context
0929 16:05:00.775460 (+ 189us) client_negotiation.cc:240] Negotiation successful
0929 16:05:00.776153 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"thread_start_us":18568,"threads_started":1}
I20260929 16:05:00.780261 28255 heartbeater.cc:347] Connected to a master server at 127.20.121.188:41459
I20260929 16:05:00.780308 28256 heartbeater.cc:347] Connected to a master server at 127.20.121.190:43697
I20260929 16:05:00.780640 28255 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:00.780645 28256 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:00.781575 28255 heartbeater.cc:511] Master 127.20.121.188:41459 requested a full tablet report, sending...
I20260929 16:05:00.782238 28256 heartbeater.cc:511] Master 127.20.121.190:43697 requested a full tablet report, sending...
I20260929 16:05:00.783401 27136 ts_manager.cc:194] Re-registered known tserver with Master: 5188495bc43c49e48ebdf870aa0b5d80 (127.20.121.130:37735)
I20260929 16:05:00.783568 27925 ts_manager.cc:194] Registered new tserver with Master: 5188495bc43c49e48ebdf870aa0b5d80 (127.20.121.130:37735)
I20260929 16:05:00.784601 27136 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:54507
I20260929 16:05:00.786144 28266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.748954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40630 (local address 127.20.121.131:33721)
0929 16:05:00.763787 (+ 14833us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:00.763801 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:00.764566 (+ 765us) server_negotiation.cc:407] Connection header received
0929 16:05:00.764640 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:00.764651 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:00.764965 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:00.765132 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:00.767469 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.769143 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.771047 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.771986 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.772174 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.778947 (+ 6773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:00.778965 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:00.778981 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:00.779030 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:00.782738 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.783633 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.783641 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.783645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.783802 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.784275 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.784280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.784284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.784672 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:00.784961 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:00.785437 (+ 476us) server_negotiation.cc:299] Negotiation successful
0929 16:05:00.785695 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14629,"thread_start_us":13317,"threads_started":1}
I20260929 16:05:00.787487 27589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:00.789057 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.748176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:57513 (local address 127.20.121.189:33763)
0929 16:05:00.764508 (+ 16332us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:00.764519 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:00.765143 (+ 624us) server_negotiation.cc:407] Connection header received
0929 16:05:00.765416 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:00.765424 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:00.765518 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:00.765601 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:00.767900 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.768978 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.770964 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.771586 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.771691 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.778556 (+ 6865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:00.778567 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:00.778574 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:00.778622 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:00.784867 (+ 6245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.785809 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.785815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.785819 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.785978 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:00.786570 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:00.786576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:00.786580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:00.786896 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:00.787060 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:00.788233 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0929 16:05:00.788556 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13673,"thread_start_us":13530,"threads_started":1}
I20260929 16:05:00.789350 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:00.747799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.20.121.130:57513)
0929 16:05:00.765036 (+ 17237us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:00.765057 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:00.765255 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:00.766214 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:00.766219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:00.767217 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:00.767719 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.767736 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.769167 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.769173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.770788 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:00.770804 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:00.771983 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:00.771989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:00.772200 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:00.774124 (+ 1924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:00.774147 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:00.778291 (+ 4144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:00.785039 (+ 6748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.785046 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.785050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.785639 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.786151 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:00.786155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:00.786157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:00.786419 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:00.787039 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:00.787044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:00.787801 (+ 757us) client_negotiation.cc:769] Sending connection context
0929 16:05:00.788025 (+ 224us) client_negotiation.cc:240] Negotiation successful
0929 16:05:00.788528 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17095,"thread_start_us":6946,"threads_started":1}
I20260929 16:05:00.790418 28257 heartbeater.cc:347] Connected to a master server at 127.20.121.189:33763
I20260929 16:05:00.790591 28257 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:00.791137 28257 heartbeater.cc:511] Master 127.20.121.189:33763 requested a full tablet report, sending...
I20260929 16:05:00.792744 27738 ts_manager.cc:194] Registered new tserver with Master: 5188495bc43c49e48ebdf870aa0b5d80 (127.20.121.130:37735)
I20260929 16:05:00.879027 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 27466
I20260929 16:05:00.895874 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:33721
--local_ip_for_outbound_sockets=127.20.121.131
--tserver_master_addrs=127.20.121.190:43697
--webserver_port=33773
--webserver_interface=127.20.121.131
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab
--principal=oryx/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:43683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.121.190:43697,127.20.121.189:33763
--tserver_master_addrs=127.20.121.190:43697,127.20.121.189:33763,127.20.121.188:41459 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:05:01.130873 28107 heartbeater.cc:503] Master 127.20.121.190:43697 was elected leader, sending a full tablet report...
W20260929 16:05:01.255839 28268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:01.256122 28268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:01.256317 28268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:01.263839 28268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:01.263965 28268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:01.264153 28268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:05:01.274927 28268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:43683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab
--principal=oryx/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:33721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=33773
--tserver_master_addrs=127.20.121.190:43697,127.20.121.189:33763,127.20.121.188:41459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:01.276451 28268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:01.278213 28268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:01.292708 28274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:01.292682 28275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:01.292742 28277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:01.293911 28268 server_base.cc:1034] running on GCE node
I20260929 16:05:01.294834 28268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:01.295974 28268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:01.297253 28268 hybrid_clock.cc:648] HybridClock initialized: now 1790697901297187 us; error 74 us; skew 500 ppm
I20260929 16:05:01.297681 28268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:01 dist-test-slave-x07d krb5kdc[27092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697901, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:01.301561 28268 init.cc:377] Logged in from keytab as oryx/127.20.121.131@KRBTEST.COM (short username oryx)
I20260929 16:05:01.303345 28268 webserver.cc:460] Webserver started at http://127.20.121.131:33773/ using document root <none> and password file <none>
I20260929 16:05:01.304092 28268 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:01.304217 28268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:01.310467 28268 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 16:05:01.313686 28284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:01.315384 28268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:05:01.315595 28268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "37e9aca105a7457c8459e7c732a12f20"
format_stamp: "Formatted at 2026-09-29 16:04:57 on dist-test-slave-x07d"
server_key: "9bb47fa958159ca462e1fa4e8ed8c674"
server_key_iv: "46fd00e2d3f3501b7bcdc457a4ead03a"
server_key_version: "encryptionkey@0"
I20260929 16:05:01.316640 28268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:01.360001 28268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:01.365168 28268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:01.365541 28268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:01.366688 28268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:01.368307 28268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:01.368412 28268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:01.368553 28268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:01.368657 28268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:01.406631 28268 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:33721
I20260929 16:05:01.406729 28396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:33721 every 8 connection(s)
I20260929 16:05:01.408226 28268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:05:01.409132 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 28268
Sep 29 16:05:01 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697901, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
Sep 29 16:05:01 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697901, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.189@KRBTEST.COM
I20260929 16:05:01.459853 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.411696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:37553 (local address 127.20.121.190:43697)
0929 16:05:01.433273 (+ 21577us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.433284 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.433310 (+ 26us) server_negotiation.cc:407] Connection header received
0929 16:05:01.433578 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.433588 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.433690 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.433836 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:01.436956 (+ 3120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.438661 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.443163 (+ 4502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.443988 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.444107 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.450764 (+ 6657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:01.450772 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:01.450778 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:01.450827 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:01.455348 (+ 4521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.456768 (+ 1420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.456774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.456778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.456956 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.457505 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.457513 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.457519 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.457941 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:01.458195 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.459075 (+ 880us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.459367 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21404,"thread_start_us":21740,"threads_started":1}
I20260929 16:05:01.463657 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.412587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.131:37553)
0929 16:05:01.423092 (+ 10505us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:01.423175 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:01.430663 (+ 7488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:01.434148 (+ 3485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:01.434172 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:01.435417 (+ 1245us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:01.436614 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.436652 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.438939 (+ 2287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.438947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.442080 (+ 3133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.442096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.444493 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.444504 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.444846 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.446298 (+ 1452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:01.446332 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:01.450444 (+ 4112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:01.455991 (+ 5547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:01.456001 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:01.456019 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:01.456593 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:01.457077 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:01.457084 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:01.457087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:01.457314 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:01.458153 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:01.458163 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:01.458607 (+ 444us) client_negotiation.cc:769] Sending connection context
0929 16:05:01.458924 (+ 317us) client_negotiation.cc:240] Negotiation successful
0929 16:05:01.459533 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10074,"thread_start_us":9769,"threads_started":1}
I20260929 16:05:01.465739 28398 heartbeater.cc:347] Connected to a master server at 127.20.121.190:43697
I20260929 16:05:01.466161 28398 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:01.467198 28398 heartbeater.cc:511] Master 127.20.121.190:43697 requested a full tablet report, sending...
I20260929 16:05:01.469882 27136 ts_manager.cc:194] Re-registered known tserver with Master: 37e9aca105a7457c8459e7c732a12f20 (127.20.121.131:33721)
I20260929 16:05:01.471395 27136 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:37553
I20260929 16:05:01.481488 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.434353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39798 (local address 127.20.121.188:41459)
0929 16:05:01.442798 (+ 8445us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.442804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.444001 (+ 1197us) server_negotiation.cc:407] Connection header received
0929 16:05:01.444102 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.444111 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.444214 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.444347 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:01.449161 (+ 4814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.450657 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.460056 (+ 9399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.461060 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.461213 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.471031 (+ 9818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:01.471043 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:01.471052 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:01.471114 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:01.475760 (+ 4646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.477147 (+ 1387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.477154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.477158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.477357 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.478039 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.478046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.478053 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.478561 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:01.478889 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.480471 (+ 1582us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.480839 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8283,"thread_start_us":9057,"threads_started":1}
I20260929 16:05:01.489132 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.431272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.20.121.131:41157)
0929 16:05:01.441574 (+ 10302us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:01.441599 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:01.441754 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:01.442421 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:01.442428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:01.443213 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:01.443561 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.443585 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.445658 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.445664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.447594 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.447614 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.448720 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.448730 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.450769 (+ 2039us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.451826 (+ 1057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:01.451869 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:01.457954 (+ 6085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:01.484526 (+ 26572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:01.484536 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:01.484543 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:01.485495 (+ 952us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:01.486201 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:01.486214 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:01.486221 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:01.486441 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:01.487719 (+ 1278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:01.487728 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:01.487962 (+ 234us) client_negotiation.cc:769] Sending connection context
0929 16:05:01.488123 (+ 161us) client_negotiation.cc:240] Negotiation successful
0929 16:05:01.488577 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10148,"thread_start_us":873,"threads_started":1}
I20260929 16:05:01.489566 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.433594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.20.121.190:43697)
0929 16:05:01.460631 (+ 27037us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.460637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.460657 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:05:01.460715 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.460724 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.460826 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.460956 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:01.463296 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.465519 (+ 2223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.472140 (+ 6621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.472927 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.473085 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.477411 (+ 4326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:01.477421 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:01.477429 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:01.477476 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:01.485446 (+ 7970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.487000 (+ 1554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.487006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.487010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.487173 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.487614 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.487619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.487622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.487983 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:01.488249 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.488617 (+ 368us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.488860 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26928,"threads_started":1}
Sep 29 16:05:01 dist-test-slave-x07d krb5kdc[27092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697901, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.188@KRBTEST.COM
I20260929 16:05:01.490540 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.424034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:41157 (local address 127.20.121.189:33763)
0929 16:05:01.440999 (+ 16965us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.441007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.441895 (+ 888us) server_negotiation.cc:407] Connection header received
0929 16:05:01.441971 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.441977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.442089 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.442200 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:01.443770 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.445074 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.447841 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.448491 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.448678 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.465561 (+ 16883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:01.465572 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:01.465581 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:01.470477 (+ 4896us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:01.484303 (+ 13826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.485724 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.485734 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.485741 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.485966 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.487027 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.487035 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.487041 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.487529 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:01.489664 (+ 2135us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.489833 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.490107 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16807,"thread_start_us":16977,"threads_started":1}
I20260929 16:05:01.491508 28399 heartbeater.cc:347] Connected to a master server at 127.20.121.189:33763
I20260929 16:05:01.491714 28399 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:01.492311 28399 heartbeater.cc:511] Master 127.20.121.189:33763 requested a full tablet report, sending...
I20260929 16:05:01.493799 27738 ts_manager.cc:194] Registered new tserver with Master: 37e9aca105a7457c8459e7c732a12f20 (127.20.121.131:33721)
I20260929 16:05:01.502223 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.443077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.20.121.189:33763)
0929 16:05:01.472334 (+ 29257us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.472341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.472364 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:05:01.472432 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.472438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.472526 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.478602 (+ 6076us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:01.478880 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.480794 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.485447 (+ 4653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.490113 (+ 4666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.490903 (+ 790us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.493949 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:01.493955 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:01.493960 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:01.493997 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:01.498328 (+ 4331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.499372 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.499379 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.499383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.499565 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.499986 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.499992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.499995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.500287 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:01.500550 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.501502 (+ 952us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.501749 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28786,"thread_start_us":28623,"threads_started":1}
I20260929 16:05:01.502847 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.411794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:40505 (local address 127.20.121.188:41459)
0929 16:05:01.430583 (+ 18789us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.430594 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.432262 (+ 1668us) server_negotiation.cc:407] Connection header received
0929 16:05:01.432448 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.432455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.432554 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.432679 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:01.454228 (+ 21549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.455685 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.481114 (+ 25429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.481883 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.482037 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.495714 (+ 13677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:01.495724 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:01.495731 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:01.495817 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:01.499372 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.500494 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.500500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.500503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.500701 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.501219 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.501224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.501227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.501571 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:01.501713 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.502240 (+ 527us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.502488 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18572,"thread_start_us":18455,"threads_started":1}
I20260929 16:05:01.503377 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.413410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:41459 (local address 127.20.121.131:40505)
0929 16:05:01.432080 (+ 18670us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:01.432117 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:01.432288 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:01.450281 (+ 17993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:01.450288 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:01.451607 (+ 1319us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:01.452111 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.452129 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.464800 (+ 12671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.464806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.470242 (+ 5436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.470264 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.482424 (+ 12160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.482431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.482635 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.485115 (+ 2480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:01.485146 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:01.495370 (+ 10224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:01.499569 (+ 4199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:01.499577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:01.499595 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:01.500320 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:01.500888 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:01.500892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:01.500896 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:01.501068 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:01.501731 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:01.501736 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:01.501915 (+ 179us) client_negotiation.cc:769] Sending connection context
0929 16:05:01.502098 (+ 183us) client_negotiation.cc:240] Negotiation successful
0929 16:05:01.502730 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18487,"thread_start_us":35440,"threads_started":1}
I20260929 16:05:01.507022 28397 heartbeater.cc:347] Connected to a master server at 127.20.121.188:41459
I20260929 16:05:01.507231 28397 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:01.507594 28397 heartbeater.cc:511] Master 127.20.121.188:41459 requested a full tablet report, sending...
I20260929 16:05:01.501730 27136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 16:05:01.509086 27925 ts_manager.cc:194] Registered new tserver with Master: 37e9aca105a7457c8459e7c732a12f20 (127.20.121.131:33721)
W20260929 16:05:01.510599 27136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:05:01.587452 28431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.535705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:37735 (local address 127.0.0.1:47740)
0929 16:05:01.537136 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:01.537162 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:01.537328 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:01.577947 (+ 40619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:01.577956 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:01.578022 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:01.578561 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.578580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.581441 (+ 2861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.581452 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.584700 (+ 3248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.584729 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.586593 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.586600 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.586777 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.586814 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 16:05:01.586914 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 16:05:01.587067 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1251,"thread_start_us":49566,"threads_started":1}
I20260929 16:05:01.588433 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.536061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47740 (local address 127.20.121.130:37735)
0929 16:05:01.576155 (+ 40094us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.576168 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.576211 (+ 43us) server_negotiation.cc:407] Connection header received
0929 16:05:01.576437 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.576450 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.576903 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.578754 (+ 1851us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:01.579034 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.581204 (+ 2170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.585022 (+ 3818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.586363 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.586599 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.586806 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.587931 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.588125 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39865,"thread_start_us":42379,"threads_started":1}
I20260929 16:05:01.592825 28190 tablet_service.cc:1467] Processing CreateTablet for tablet 69613475743e410ca3b7d206b0b58be6 (DEFAULT_TABLE table=test-table [id=8821c00be6454b12a31f7e082084089f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:01.594503 28190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69613475743e410ca3b7d206b0b58be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:01.596203 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.536484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.20.121.129:44627)
0929 16:05:01.577760 (+ 41276us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.577772 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.586899 (+ 9127us) server_negotiation.cc:407] Connection header received
0929 16:05:01.587011 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.587019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.587241 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.587449 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:01.588069 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.589695 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.593507 (+ 3812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.595057 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.595244 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.595443 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.595681 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.595848 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40970,"thread_start_us":41644,"threads_started":1}
I20260929 16:05:01.596539 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.535968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:44627 (local address 127.0.0.1:60628)
0929 16:05:01.581939 (+ 45971us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:01.581961 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:01.582183 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:01.587464 (+ 5281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:01.587469 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:01.587498 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:01.587867 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.587878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.590895 (+ 3017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.590901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.593228 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.593247 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.595294 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.595301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.595479 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.595518 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 16:05:01.595637 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 16:05:01.595790 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45825,"mutex_wait_us":56,"thread_start_us":45675,"threads_started":1}
I20260929 16:05:01.597815 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.536451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:33721 (local address 127.0.0.1:40640)
0929 16:05:01.585519 (+ 49068us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:01.585535 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:01.585702 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:01.587980 (+ 2278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:01.587984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:01.588013 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:01.588435 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.588447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.591871 (+ 3424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.591877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.594065 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.594080 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.596981 (+ 2901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.596988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.597171 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.597203 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 16:05:01.597297 (+ 94us) client_negotiation.cc:240] Negotiation successful
0929 16:05:01.597437 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48961,"mutex_wait_us":1,"thread_start_us":1560,"threads_started":1}
I20260929 16:05:01.597879 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.537018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.20.121.131:33721)
0929 16:05:01.575348 (+ 38330us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.575364 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.585668 (+ 10304us) server_negotiation.cc:407] Connection header received
0929 16:05:01.585882 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.585893 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.586635 (+ 742us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.586836 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:01.589883 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.591645 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.594243 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.596313 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.596544 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.596727 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.597379 (+ 652us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.597549 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38035,"thread_start_us":38385,"threads_started":1}
I20260929 16:05:01.600660 28041 tablet_service.cc:1467] Processing CreateTablet for tablet 69613475743e410ca3b7d206b0b58be6 (DEFAULT_TABLE table=test-table [id=8821c00be6454b12a31f7e082084089f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:01.602346 28041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69613475743e410ca3b7d206b0b58be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:01.601816 28332 tablet_service.cc:1467] Processing CreateTablet for tablet 69613475743e410ca3b7d206b0b58be6 (DEFAULT_TABLE table=test-table [id=8821c00be6454b12a31f7e082084089f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:01.603507 28332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69613475743e410ca3b7d206b0b58be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:01.609323 28434 tablet_bootstrap.cc:492] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: Bootstrap starting.
I20260929 16:05:01.613207 28434 tablet_bootstrap.cc:654] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:01.614617 28434 log.cc:824] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:01.617622 28434 tablet_bootstrap.cc:492] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: No bootstrap required, opened a new log
I20260929 16:05:01.618462 28434 ts_tablet_manager.cc:1397] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 16:05:01.626431 28437 tablet_bootstrap.cc:492] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: Bootstrap starting.
I20260929 16:05:01.626503 28434 raft_consensus.cc:348] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:01.627038 28434 raft_consensus.cc:374] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:01.627166 28434 raft_consensus.cc:729] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5188495bc43c49e48ebdf870aa0b5d80, State: Initialized, Role: FOLLOWER
I20260929 16:05:01.627970 28434 consensus_queue.cc:260] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:01.628369 28436 tablet_bootstrap.cc:492] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Bootstrap starting.
I20260929 16:05:01.630903 28437 tablet_bootstrap.cc:654] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:01.632050 28437 log.cc:824] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:01.633464 28436 tablet_bootstrap.cc:654] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:01.634395 28434 ts_tablet_manager.cc:1428] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260929 16:05:01.634761 28256 heartbeater.cc:503] Master 127.20.121.190:43697 was elected leader, sending a full tablet report...
I20260929 16:05:01.634881 28436 log.cc:824] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:01.639798 28437 tablet_bootstrap.cc:492] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: No bootstrap required, opened a new log
I20260929 16:05:01.640234 28437 ts_tablet_manager.cc:1397] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260929 16:05:01.643399 28438 raft_consensus.cc:482] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:05:01.643802 28438 raft_consensus.cc:504] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:01.644290 28436 tablet_bootstrap.cc:492] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: No bootstrap required, opened a new log
I20260929 16:05:01.644616 28436 ts_tablet_manager.cc:1397] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.005s
I20260929 16:05:01.650833 28437 raft_consensus.cc:348] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:01.651418 28437 raft_consensus.cc:374] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:01.651538 28437 raft_consensus.cc:729] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e9aca105a7457c8459e7c732a12f20, State: Initialized, Role: FOLLOWER
I20260929 16:05:01.652334 28437 consensus_queue.cc:260] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:01.652972 28438 leader_election.cc:288] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627), 37e9aca105a7457c8459e7c732a12f20 (127.20.121.131:33721)
I20260929 16:05:01.657986 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.646178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:39087 (local address 127.20.121.129:44627)
0929 16:05:01.646403 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.646427 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.646832 (+ 405us) server_negotiation.cc:407] Connection header received
0929 16:05:01.647539 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.647546 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.647655 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.647716 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:01.648540 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.649797 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.653321 (+ 3524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.655261 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.655542 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.655666 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.657218 (+ 1552us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.657542 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 16:05:01.659529 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.646093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:44627 (local address 127.20.121.130:39087)
0929 16:05:01.646713 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:01.646734 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:01.646952 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:01.647948 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:01.647953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:01.647984 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:01.648352 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.648365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.650052 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.650057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.652913 (+ 2856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.652945 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.656531 (+ 3586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.656541 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.656910 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.656972 (+ 62us) client_negotiation.cc:769] Sending connection context
0929 16:05:01.658703 (+ 1731us) client_negotiation.cc:240] Negotiation successful
0929 16:05:01.658976 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":307,"threads_started":1}
I20260929 16:05:01.661222 28061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69613475743e410ca3b7d206b0b58be6" candidate_uuid: "5188495bc43c49e48ebdf870aa0b5d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" is_pre_election: true
W20260929 16:05:01.663054 28143 leader_election.cc:341] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627): Illegal state: must be running to vote when last-logged opid is not known
I20260929 16:05:01.663586 28436 raft_consensus.cc:348] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:01.664016 28436 raft_consensus.cc:374] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:01.664088 28436 raft_consensus.cc:729] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1600f1ed9e3c4cda9feac1f01ae5d3e0, State: Initialized, Role: FOLLOWER
I20260929 16:05:01.664726 28436 consensus_queue.cc:260] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:01.671154 28436 ts_tablet_manager.cc:1428] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Time spent starting tablet: real 0.026s user 0.017s sys 0.003s
I20260929 16:05:01.675158 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.647253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:54689 (local address 127.20.121.131:33721)
0929 16:05:01.647430 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.647436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.660998 (+ 13562us) server_negotiation.cc:407] Connection header received
0929 16:05:01.661349 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.661357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.661484 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.661571 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:01.663085 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.664772 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.671803 (+ 7031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.673711 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.673899 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.674037 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.674817 (+ 780us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.674963 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 16:05:01.675460 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.647001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:33721 (local address 127.20.121.130:54689)
0929 16:05:01.660867 (+ 13866us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:01.660892 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:01.661147 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:01.662029 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:01.662037 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:01.662097 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:01.662409 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.662425 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.665231 (+ 2806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.665240 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.671450 (+ 6210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:01.671484 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.674250 (+ 2766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.674260 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:01.674540 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.674612 (+ 72us) client_negotiation.cc:769] Sending connection context
0929 16:05:01.674727 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 16:05:01.674965 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13683,"thread_start_us":16659,"threads_started":1}
I20260929 16:05:01.677011 28352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69613475743e410ca3b7d206b0b58be6" candidate_uuid: "5188495bc43c49e48ebdf870aa0b5d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e9aca105a7457c8459e7c732a12f20" is_pre_election: true
I20260929 16:05:01.677399 28437 ts_tablet_manager.cc:1428] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: Time spent starting tablet: real 0.037s user 0.016s sys 0.020s
I20260929 16:05:01.677726 28398 heartbeater.cc:503] Master 127.20.121.190:43697 was elected leader, sending a full tablet report...
I20260929 16:05:01.677660 28352 raft_consensus.cc:2454] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5188495bc43c49e48ebdf870aa0b5d80 in term 0.
W20260929 16:05:01.677918 28405 tablet.cc:2369] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:05:01.678447 28144 leader_election.cc:302] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 37e9aca105a7457c8459e7c732a12f20, 5188495bc43c49e48ebdf870aa0b5d80; no voters: 1600f1ed9e3c4cda9feac1f01ae5d3e0
I20260929 16:05:01.678907 28438 raft_consensus.cc:2781] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:05:01.679054 28438 raft_consensus.cc:482] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:05:01.679152 28438 raft_consensus.cc:3037] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:01.682437 28438 raft_consensus.cc:504] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:01.683094 28438 leader_election.cc:288] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [CANDIDATE]: Term 1 election: Requested vote from peers 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627), 37e9aca105a7457c8459e7c732a12f20 (127.20.121.131:33721)
I20260929 16:05:01.683650 28061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69613475743e410ca3b7d206b0b58be6" candidate_uuid: "5188495bc43c49e48ebdf870aa0b5d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0"
I20260929 16:05:01.683681 28352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69613475743e410ca3b7d206b0b58be6" candidate_uuid: "5188495bc43c49e48ebdf870aa0b5d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e9aca105a7457c8459e7c732a12f20"
I20260929 16:05:01.683853 28352 raft_consensus.cc:3037] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:01.683950 28061 raft_consensus.cc:3037] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:01.686378 28061 raft_consensus.cc:2454] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5188495bc43c49e48ebdf870aa0b5d80 in term 1.
I20260929 16:05:01.686655 28352 raft_consensus.cc:2454] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5188495bc43c49e48ebdf870aa0b5d80 in term 1.
I20260929 16:05:01.686919 28143 leader_election.cc:302] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1600f1ed9e3c4cda9feac1f01ae5d3e0, 5188495bc43c49e48ebdf870aa0b5d80; no voters:
I20260929 16:05:01.687288 28438 raft_consensus.cc:2781] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:01.687793 28438 raft_consensus.cc:686] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 1 LEADER]: Becoming Leader. State: Replica: 5188495bc43c49e48ebdf870aa0b5d80, State: Running, Role: LEADER
I20260929 16:05:01.688410 28438 consensus_queue.cc:237] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:01.695240 27136 catalog_manager.cc:5495] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5188495bc43c49e48ebdf870aa0b5d80 (127.20.121.130). New cstate: current_term: 1 leader_uuid: "5188495bc43c49e48ebdf870aa0b5d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } health_report { overall_health: UNKNOWN } } }
I20260929 16:05:01.745712 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.738143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47756 (local address 127.20.121.130:37735)
0929 16:05:01.738331 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.738335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.738354 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:05:01.738444 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.738449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.738549 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.738687 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:01.739078 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.740108 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.742534 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.743116 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.743278 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.744364 (+ 1086us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:01.744931 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:01.745119 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.745236 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.745317 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260929 16:05:01.747128 28258 tablet.cc:2369] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:05:01.755972 28061 raft_consensus.cc:1261] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 1 FOLLOWER]: Refusing update from remote peer 5188495bc43c49e48ebdf870aa0b5d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:01.756108 28352 raft_consensus.cc:1261] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20 [term 1 FOLLOWER]: Refusing update from remote peer 5188495bc43c49e48ebdf870aa0b5d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:01.757120 28445 consensus_queue.cc:1035] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:01.757613 28438 consensus_queue.cc:1035] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:01.773253 28449 mvcc.cc:205] Tried to move back new op lower bound from 7334698605580767232 to 7334698605320392704. Current Snapshot: MvccSnapshot[applied={T|T < 7334698605580767232}]
I20260929 16:05:01.798938 27137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:01.799249 27137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:01.806788 27137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7adf02d0479447608d50d296296b2eaa: Sending DeleteTablet for 3 replicas of tablet 69613475743e410ca3b7d206b0b58be6
I20260929 16:05:01.808032 28190 tablet_service.cc:1514] Processing DeleteTablet for tablet 69613475743e410ca3b7d206b0b58be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:01 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:47740
I20260929 16:05:01.808557 28041 tablet_service.cc:1514] Processing DeleteTablet for tablet 69613475743e410ca3b7d206b0b58be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:01 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:60628
I20260929 16:05:01.809335 28459 tablet_replica.cc:330] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: stopping tablet replica
I20260929 16:05:01.809253 28332 tablet_service.cc:1514] Processing DeleteTablet for tablet 69613475743e410ca3b7d206b0b58be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:01 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:40640
I20260929 16:05:01.809741 28459 raft_consensus.cc:2229] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:05:01.810240 28461 tablet_replica.cc:330] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: stopping tablet replica
I20260929 16:05:01.810642 28459 raft_consensus.cc:2258] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:01.810688 28461 raft_consensus.cc:2229] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:01.811539 28461 raft_consensus.cc:2258] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:01.814345 28459 ts_tablet_manager.cc:1905] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:01.815148 28461 ts_tablet_manager.cc:1905] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:01.820737 28460 tablet_replica.cc:330] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: stopping tablet replica
I20260929 16:05:01.821437 28460 raft_consensus.cc:2229] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:01.822095 28460 raft_consensus.cc:2258] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:01.823348 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 27105
I20260929 16:05:01.823891 28459 ts_tablet_manager.cc:1918] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:01.824100 28459 log.cc:1196] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/69613475743e410ca3b7d206b0b58be6
I20260929 16:05:01.824918 27167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:01.827688 28461 ts_tablet_manager.cc:1918] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:01.828043 28461 log.cc:1196] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/69613475743e410ca3b7d206b0b58be6
I20260929 16:05:01.828850 28461 ts_tablet_manager.cc:1939] T 69613475743e410ca3b7d206b0b58be6 P 37e9aca105a7457c8459e7c732a12f20: Deleting consensus metadata
I20260929 16:05:01.830762 28459 ts_tablet_manager.cc:1939] T 69613475743e410ca3b7d206b0b58be6 P 5188495bc43c49e48ebdf870aa0b5d80: Deleting consensus metadata
I20260929 16:05:01.830892 28460 ts_tablet_manager.cc:1905] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:01.842685 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.812337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54888 (local address 127.20.121.189:33763)
0929 16:05:01.812589 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.812597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.820425 (+ 7828us) server_negotiation.cc:407] Connection header received
0929 16:05:01.820674 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.820680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.820793 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.820931 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:01.825361 (+ 4430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.826562 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.828744 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.829525 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.829645 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.833803 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:01.833811 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:01.833817 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:01.833864 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:01.838647 (+ 4783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.839738 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.839744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.839747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.839908 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.840595 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.840602 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.840606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.841018 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:01.841294 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.841971 (+ 677us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.842266 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260929 16:05:01.850860 28460 ts_tablet_manager.cc:1918] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:01.851099 28460 log.cc:1196] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/69613475743e410ca3b7d206b0b58be6
I20260929 16:05:01.851775 28460 ts_tablet_manager.cc:1939] T 69613475743e410ca3b7d206b0b58be6 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Deleting consensus metadata
I20260929 16:05:01.865567 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:01.821622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.20.121.188:41459)
0929 16:05:01.821790 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:01.821796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:01.845285 (+ 23489us) server_negotiation.cc:407] Connection header received
0929 16:05:01.845576 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:01.845584 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:01.845694 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:01.845776 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:01.847838 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.849051 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.851416 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:01.852669 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:01.852793 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:01.858213 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:01.858221 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:01.858226 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:01.858277 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:01.862041 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.863116 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.863124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.863128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.863288 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:01.863766 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:01.863772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:01.863775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:01.864156 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:01.864388 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:01.864963 (+ 575us) server_negotiation.cc:299] Negotiation successful
0929 16:05:01.865205 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 16:05:01.983397 27121 catalog_manager.cc:4841] TS 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627): tablet 69613475743e410ca3b7d206b0b58be6 (table test-table [id=8821c00be6454b12a31f7e082084089f]) successfully deleted
I20260929 16:05:01.983847 27123 catalog_manager.cc:4841] TS 5188495bc43c49e48ebdf870aa0b5d80 (127.20.121.130:37735): tablet 69613475743e410ca3b7d206b0b58be6 (table test-table [id=8821c00be6454b12a31f7e082084089f]) successfully deleted
I20260929 16:05:01.983925 27122 catalog_manager.cc:4841] TS 37e9aca105a7457c8459e7c732a12f20 (127.20.121.131:33721): tablet 69613475743e410ca3b7d206b0b58be6 (table test-table [id=8821c00be6454b12a31f7e082084089f]) successfully deleted
W20260929 16:05:01.984050 27123 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.20.121.130:37735, user_credentials={real_user=slave}} blocked reactor thread for 151261us
I20260929 16:05:01.984542 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 27105
I20260929 16:05:02.033761 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:02.014992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54904 (local address 127.20.121.189:33763)
0929 16:05:02.015171 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:02.015178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:02.015201 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:05:02.015392 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:02.015401 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:02.015497 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:02.015628 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:02.017689 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:02.018984 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:02.021417 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:02.022094 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:02.022245 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:02.026297 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:02.026304 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:02.026309 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:02.026371 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:02.030440 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:02.031390 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:02.031397 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:02.031400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:02.031582 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:02.032108 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:02.032114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:02.032117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:02.032504 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:02.032743 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:02.033174 (+ 431us) server_negotiation.cc:299] Negotiation successful
0929 16:05:02.033402 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 16:05:02.035528 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:02.013247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.20.121.188:41459)
0929 16:05:02.013579 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:02.013589 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:02.015425 (+ 1836us) server_negotiation.cc:407] Connection header received
0929 16:05:02.015629 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:02.015635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:02.015717 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:02.015817 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:02.018247 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:02.019362 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:02.022278 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:02.022944 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:02.023100 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:02.026077 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:02.026084 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:02.026090 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:02.026165 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:02.030418 (+ 4253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:02.032080 (+ 1662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:02.032089 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:02.032092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:02.032232 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:02.032691 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:02.032697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:02.032700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:02.033154 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:02.033401 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:02.034730 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0929 16:05:02.034946 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260929 16:05:02.787446 28470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:02.786347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.131:46263)
0929 16:05:02.787044 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:02.787209 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":459,"threads_started":1}
W20260929 16:05:02.787947 28398 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:43697 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
I20260929 16:05:02.788056 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:02.786629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.130:44177)
0929 16:05:02.787475 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:02.787848 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":735,"thread_start_us":654,"threads_started":1}
W20260929 16:05:02.788489 28256 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:43697 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
I20260929 16:05:02.788658 28472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:02.787720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.129:35903)
0929 16:05:02.788304 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:02.788426 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":558,"threads_started":1}
W20260929 16:05:02.789064 28107 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:43697 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
I20260929 16:05:03.306888 28473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7adf02d0479447608d50d296296b2eaa)
I20260929 16:05:03.307304 28473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } }
I20260929 16:05:03.307646 28474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7adf02d0479447608d50d296296b2eaa)
I20260929 16:05:03.307976 28474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } }
I20260929 16:05:03.309283 28473 leader_election.cc:288] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697), 0c561530e7bf44299e555f50c33f5317 (127.20.121.189:33763)
I20260929 16:05:03.310895 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.309501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56448)
0929 16:05:03.310344 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.310543 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":536,"threads_started":1}
I20260929 16:05:03.310952 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.309723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56456)
0929 16:05:03.310265 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.310439 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":312,"threads_started":1}
I20260929 16:05:03.312331 28474 leader_election.cc:288] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697), 30c111a79abb4f979bd085b0ac6c3579 (127.20.121.188:41459)
W20260929 16:05:03.312361 27909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
W20260929 16:05:03.312469 27722 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
I20260929 16:05:03.315321 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.314327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56460)
0929 16:05:03.314711 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.314974 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
W20260929 16:05:03.315608 27722 leader_election.cc:334] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697): Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
I20260929 16:05:03.319783 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.319131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56462)
0929 16:05:03.319344 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.319519 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260929 16:05:03.320115 27909 leader_election.cc:334] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697): Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
I20260929 16:05:03.321015 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.310653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:41459 (local address 127.0.0.1:39824)
0929 16:05:03.311182 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.311203 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:03.311354 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:03.313672 (+ 2318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:03.313684 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:03.313720 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:03.314250 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:03.314273 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:03.315991 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:03.315996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:03.317410 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:03.317422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:03.320034 (+ 2612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:03.320042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:03.320247 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:03.320294 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 16:05:03.320434 (+ 140us) client_negotiation.cc:240] Negotiation successful
0929 16:05:03.320629 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":38,"thread_start_us":949,"threads_started":1}
I20260929 16:05:03.321653 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.311271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.20.121.188:41459)
0929 16:05:03.313253 (+ 1982us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:03.313259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:03.313278 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:05:03.313339 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:03.313345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:03.313428 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:03.313605 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:03.314504 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:03.315757 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:03.317597 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:03.319812 (+ 2215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:03.320823 (+ 1011us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:03.320935 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:03.320996 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 16:05:03.321177 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":1761,"thread_start_us":1769,"threads_started":1}
I20260929 16:05:03.321687 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.310520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33763 (local address 127.0.0.1:54906)
0929 16:05:03.311443 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.311470 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:03.311672 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:03.312922 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:03.312930 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:03.313004 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:03.313328 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:03.313339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:03.315752 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:03.315758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:03.318812 (+ 3054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:03.318826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:03.320574 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:03.320580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:03.320812 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:03.320889 (+ 77us) client_negotiation.cc:769] Sending connection context
0929 16:05:03.321065 (+ 176us) client_negotiation.cc:240] Negotiation successful
0929 16:05:03.321241 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":772,"mutex_wait_us":63,"thread_start_us":1459,"threads_started":1}
I20260929 16:05:03.321887 28480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.311874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54906 (local address 127.20.121.189:33763)
0929 16:05:03.312461 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:03.312467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:03.312491 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:05:03.312577 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:03.312581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:03.312662 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:03.312839 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:03.313730 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:03.315562 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:03.319023 (+ 3461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:03.320371 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:03.320553 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:03.320653 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:03.321108 (+ 455us) server_negotiation.cc:299] Negotiation successful
0929 16:05:03.321383 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":464,"thread_start_us":296,"threads_started":1}
I20260929 16:05:03.322494 27935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c561530e7bf44299e555f50c33f5317" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "30c111a79abb4f979bd085b0ac6c3579" is_pre_election: true
I20260929 16:05:03.322639 27748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30c111a79abb4f979bd085b0ac6c3579" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c561530e7bf44299e555f50c33f5317" is_pre_election: true
I20260929 16:05:03.322985 27748 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30c111a79abb4f979bd085b0ac6c3579 in term 1.
I20260929 16:05:03.322986 27935 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c561530e7bf44299e555f50c33f5317 in term 1.
I20260929 16:05:03.323648 27909 leader_election.cc:302] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c561530e7bf44299e555f50c33f5317, 30c111a79abb4f979bd085b0ac6c3579; no voters: 7adf02d0479447608d50d296296b2eaa
I20260929 16:05:03.323653 27723 leader_election.cc:302] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c561530e7bf44299e555f50c33f5317, 30c111a79abb4f979bd085b0ac6c3579; no voters: 7adf02d0479447608d50d296296b2eaa
I20260929 16:05:03.324082 28473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 16:05:03.324272 28473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7adf02d0479447608d50d296296b2eaa)
I20260929 16:05:03.324337 28474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 16:05:03.324388 28473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 1 FOLLOWER]: Advancing to term 2
I20260929 16:05:03.324502 28474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7adf02d0479447608d50d296296b2eaa)
I20260929 16:05:03.324632 28474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 1 FOLLOWER]: Advancing to term 2
I20260929 16:05:03.326895 28473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } }
I20260929 16:05:03.327162 28474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } }
I20260929 16:05:03.327701 28473 leader_election.cc:288] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 2 election: Requested vote from peers 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697), 0c561530e7bf44299e555f50c33f5317 (127.20.121.189:33763)
I20260929 16:05:03.328238 28474 leader_election.cc:288] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [CANDIDATE]: Term 2 election: Requested vote from peers 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697), 30c111a79abb4f979bd085b0ac6c3579 (127.20.121.188:41459)
I20260929 16:05:03.328265 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.327824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56472)
0929 16:05:03.327966 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.328063 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 16:05:03.328631 27935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c561530e7bf44299e555f50c33f5317" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "30c111a79abb4f979bd085b0ac6c3579"
I20260929 16:05:03.328783 27748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30c111a79abb4f979bd085b0ac6c3579" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c561530e7bf44299e555f50c33f5317"
I20260929 16:05:03.328975 27935 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0c561530e7bf44299e555f50c33f5317 in current term 2: Already voted for candidate 30c111a79abb4f979bd085b0ac6c3579 in this term.
I20260929 16:05:03.329010 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.328208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56476)
0929 16:05:03.328688 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.328816 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324}
I20260929 16:05:03.329167 27748 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 30c111a79abb4f979bd085b0ac6c3579 in current term 2: Already voted for candidate 0c561530e7bf44299e555f50c33f5317 in this term.
I20260929 16:05:03.329420 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.329028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56492)
0929 16:05:03.329215 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.329293 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 16:05:03.329671 27909 leader_election.cc:334] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697): Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
I20260929 16:05:03.331110 27909 leader_election.cc:302] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30c111a79abb4f979bd085b0ac6c3579; no voters: 0c561530e7bf44299e555f50c33f5317, 7adf02d0479447608d50d296296b2eaa
I20260929 16:05:03.331424 28473 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260929 16:05:03.332211 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.331616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56512)
0929 16:05:03.331932 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.332009 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
W20260929 16:05:03.332588 27722 leader_election.cc:334] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697): Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
I20260929 16:05:03.332742 27722 leader_election.cc:302] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c561530e7bf44299e555f50c33f5317; no voters: 30c111a79abb4f979bd085b0ac6c3579, 7adf02d0479447608d50d296296b2eaa
I20260929 16:05:03.333052 28474 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260929 16:05:03.790449 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.788742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.131:34455)
0929 16:05:03.789871 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.790070 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":317,"threads_started":1}
I20260929 16:05:03.791951 28485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.789799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.129:44651)
0929 16:05:03.791602 (+ 1803us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.791718 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1633,"thread_start_us":1475,"threads_started":1}
I20260929 16:05:03.792111 28484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:03.789452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.130:36539)
0929 16:05:03.791804 (+ 2352us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:03.791900 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2227,"thread_start_us":2094,"threads_started":1}
I20260929 16:05:04.793181 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.791965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.131:33929)
0929 16:05:04.792793 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.792938 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":524,"threads_started":1}
I20260929 16:05:04.795841 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.793211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.130:37643)
0929 16:05:04.795538 (+ 2327us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.795644 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2221,"thread_start_us":2120,"threads_started":1}
I20260929 16:05:04.796975 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.792959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.129:43247)
0929 16:05:04.796715 (+ 3756us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.796815 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3636,"thread_start_us":3510,"threads_started":1}
I20260929 16:05:04.863647 28489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:05:04.863870 28489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } }
I20260929 16:05:04.864748 28489 leader_election.cc:288] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697), 0c561530e7bf44299e555f50c33f5317 (127.20.121.189:33763)
I20260929 16:05:04.865995 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.864921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56740)
0929 16:05:04.865545 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.865718 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":277,"threads_started":1}
I20260929 16:05:04.866205 27748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30c111a79abb4f979bd085b0ac6c3579" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c561530e7bf44299e555f50c33f5317" is_pre_election: true
I20260929 16:05:04.866513 27748 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30c111a79abb4f979bd085b0ac6c3579 in term 2.
I20260929 16:05:04.867192 27909 leader_election.cc:302] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c561530e7bf44299e555f50c33f5317, 30c111a79abb4f979bd085b0ac6c3579; no voters:
I20260929 16:05:04.867491 28489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260929 16:05:04.867636 28489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:05:04.867779 28489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 2 FOLLOWER]: Advancing to term 3
I20260929 16:05:04.868105 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.867691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56754)
0929 16:05:04.867864 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.867966 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 16:05:04.868309 27909 leader_election.cc:334] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697): Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
I20260929 16:05:04.869956 28489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } }
I20260929 16:05:04.870743 28489 leader_election.cc:288] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 3 election: Requested vote from peers 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697), 0c561530e7bf44299e555f50c33f5317 (127.20.121.189:33763)
I20260929 16:05:04.872098 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.871394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56772)
0929 16:05:04.871790 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.871894 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265}
I20260929 16:05:04.872062 27748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30c111a79abb4f979bd085b0ac6c3579" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c561530e7bf44299e555f50c33f5317"
I20260929 16:05:04.872354 27748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 2 FOLLOWER]: Advancing to term 3
I20260929 16:05:04.873188 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.872736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56786)
0929 16:05:04.872903 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.872989 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 16:05:04.873333 27909 leader_election.cc:334] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697): Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
I20260929 16:05:04.875383 27748 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30c111a79abb4f979bd085b0ac6c3579 in term 3.
I20260929 16:05:04.876137 27909 leader_election.cc:302] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c561530e7bf44299e555f50c33f5317, 30c111a79abb4f979bd085b0ac6c3579; no voters: 7adf02d0479447608d50d296296b2eaa
I20260929 16:05:04.876588 28489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 16:05:04.877315 28489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [term 3 LEADER]: Becoming Leader. State: Replica: 30c111a79abb4f979bd085b0ac6c3579, State: Running, Role: LEADER
I20260929 16:05:04.877842 28489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } }
I20260929 16:05:04.880431 28492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30c111a79abb4f979bd085b0ac6c3579. Latest consensus state: current_term: 3 leader_uuid: "30c111a79abb4f979bd085b0ac6c3579" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:04.880771 28492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:04.881495 28494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:05:04.885258 28494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:05:04.886216 28494 catalog_manager.cc:1259] Loaded cluster ID: 635f15c821ad4c669fd2eadd14d2dfb2
I20260929 16:05:04.886292 28494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:05:04.887466 28494 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:05:04.888293 28494 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Loaded TSK: 0
I20260929 16:05:04.889194 28494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:05:04.960862 27924 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39812:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:05:04.969869 27924 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:05:04.975694 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.975190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56806)
0929 16:05:04.975403 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.975544 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260929 16:05:04.975863 27748 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [term 3 FOLLOWER]: Refusing update from remote peer 30c111a79abb4f979bd085b0ac6c3579: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260929 16:05:04.976861 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.976421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56812)
0929 16:05:04.976632 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.976716 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 16:05:04.976992 28492 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260929 16:05:04.977398 27909 consensus_peers.cc:487] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 -> Peer 7adf02d0479447608d50d296296b2eaa (127.20.121.190:43697): Couldn't send request to peer 7adf02d0479447608d50d296296b2eaa. Status: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 16:05:04.980886 28496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30c111a79abb4f979bd085b0ac6c3579. Latest consensus state: current_term: 3 leader_uuid: "30c111a79abb4f979bd085b0ac6c3579" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:04.981334 28496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:05:04.983156 28492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "30c111a79abb4f979bd085b0ac6c3579" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:04.983487 28492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579 [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:04.987542 28496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "30c111a79abb4f979bd085b0ac6c3579" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7adf02d0479447608d50d296296b2eaa" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 43697 } } peers { permanent_uuid: "0c561530e7bf44299e555f50c33f5317" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33763 } attrs { promote: false } } peers { permanent_uuid: "30c111a79abb4f979bd085b0ac6c3579" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 41459 } attrs { promote: false } } }
I20260929 16:05:04.987906 28496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c561530e7bf44299e555f50c33f5317 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:05:05.005179 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.994231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:37735 (local address 127.0.0.1:47760)
0929 16:05:04.994371 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.994390 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:04.994521 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:04.995808 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:04.995816 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:04.995852 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:04.996313 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:04.996328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:04.998438 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:04.998447 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:05.001352 (+ 2905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:05.001380 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.003922 (+ 2542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.003933 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:05.004216 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.004283 (+ 67us) client_negotiation.cc:769] Sending connection context
0929 16:05:05.004374 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 16:05:05.004611 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"spinlock_wait_cycles":24448}
I20260929 16:05:05.007077 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.994681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.20.121.130:37735)
0929 16:05:04.995463 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:04.995470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:04.995489 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:05:04.995572 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:04.995577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:04.995647 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:04.996507 (+ 860us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:04.996757 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:04.998236 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.001726 (+ 3490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.003562 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.006139 (+ 2577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.006268 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:05.006344 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 16:05:05.006574 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":496,"threads_started":1}
I20260929 16:05:05.008275 28190 tablet_service.cc:1467] Processing CreateTablet for tablet 70e263cfd1a844b9a413a4245598f960 (DEFAULT_TABLE table=test-table [id=9492079194134c18b8afbe792540258a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:05.008749 28190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70e263cfd1a844b9a413a4245598f960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:05.014963 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.996370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40642 (local address 127.20.121.131:33721)
0929 16:05:05.007218 (+ 10848us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:05.007224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:05.007252 (+ 28us) server_negotiation.cc:407] Connection header received
0929 16:05:05.007310 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:05.007313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:05.007385 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:05.007517 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:05.008066 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.009708 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.012565 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.013700 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.013890 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.013967 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:05.014471 (+ 504us) server_negotiation.cc:299] Negotiation successful
0929 16:05:05.014623 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10721,"spinlock_wait_cycles":11008,"thread_start_us":10698,"threads_started":1}
I20260929 16:05:05.015007 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.995795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:33721 (local address 127.0.0.1:40642)
0929 16:05:05.005336 (+ 9541us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.005367 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:05.005564 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:05.007538 (+ 1974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:05.007543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:05.007571 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:05.007887 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:05.007898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.009953 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.009960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:05.012363 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:05.012378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.013897 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.013903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:05.014242 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.014318 (+ 76us) client_negotiation.cc:769] Sending connection context
0929 16:05:05.014430 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 16:05:05.014603 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9330,"thread_start_us":9911,"threads_started":1}
I20260929 16:05:05.015921 28507 tablet_bootstrap.cc:492] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80: Bootstrap starting.
I20260929 16:05:05.016604 28332 tablet_service.cc:1467] Processing CreateTablet for tablet 70e263cfd1a844b9a413a4245598f960 (DEFAULT_TABLE table=test-table [id=9492079194134c18b8afbe792540258a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:05.017074 28332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70e263cfd1a844b9a413a4245598f960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:05.017568 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.995198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:44627 (local address 127.0.0.1:60630)
0929 16:05:04.995787 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:04.995804 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:04.995918 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:05.011096 (+ 15178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:05.011102 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:05.011137 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:05.011510 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:05.011520 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.012638 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.012643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:05.014918 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:05.014933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.016706 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.016713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:05.016910 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.016945 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 16:05:05.017031 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 16:05:05.017171 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":34,"thread_start_us":1076,"threads_started":1}
I20260929 16:05:05.017938 28507 tablet_bootstrap.cc:654] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:05.018301 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:04.995629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60630 (local address 127.20.121.129:44627)
0929 16:05:05.010775 (+ 15146us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:05.010779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:05.010795 (+ 16us) server_negotiation.cc:407] Connection header received
0929 16:05:05.010854 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:05.010857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:05.010930 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:05.011069 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:05.011693 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.012505 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.015125 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.016303 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.016486 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.016565 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:05.017798 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0929 16:05:05.017932 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15033,"thread_start_us":15592,"threads_started":1}
I20260929 16:05:05.019107 28041 tablet_service.cc:1467] Processing CreateTablet for tablet 70e263cfd1a844b9a413a4245598f960 (DEFAULT_TABLE table=test-table [id=9492079194134c18b8afbe792540258a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:05.019565 28041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70e263cfd1a844b9a413a4245598f960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:05.020728 28507 tablet_bootstrap.cc:492] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80: No bootstrap required, opened a new log
I20260929 16:05:05.020862 28507 ts_tablet_manager.cc:1397] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 16:05:05.021667 28507 raft_consensus.cc:348] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:05.021919 28507 raft_consensus.cc:374] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:05.021999 28507 raft_consensus.cc:729] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5188495bc43c49e48ebdf870aa0b5d80, State: Initialized, Role: FOLLOWER
I20260929 16:05:05.022271 28507 consensus_queue.cc:260] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:05.024333 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.023470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.130:43647)
0929 16:05:05.023989 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.024129 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406}
I20260929 16:05:05.025031 28507 ts_tablet_manager.cc:1428] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260929 16:05:05.025851 28510 tablet_bootstrap.cc:492] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20: Bootstrap starting.
I20260929 16:05:05.026626 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.025552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.130:50153)
0929 16:05:05.025880 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.025968 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220}
I20260929 16:05:05.027262 28255 heartbeater.cc:503] Master 127.20.121.188:41459 was elected leader, sending a full tablet report...
I20260929 16:05:05.027724 28510 tablet_bootstrap.cc:654] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:05.029855 28510 tablet_bootstrap.cc:492] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20: No bootstrap required, opened a new log
I20260929 16:05:05.030011 28510 ts_tablet_manager.cc:1397] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 16:05:05.030673 28512 tablet_bootstrap.cc:492] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Bootstrap starting.
I20260929 16:05:05.030706 28510 raft_consensus.cc:348] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:05.030980 28510 raft_consensus.cc:374] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:05.031082 28510 raft_consensus.cc:729] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e9aca105a7457c8459e7c732a12f20, State: Initialized, Role: FOLLOWER
I20260929 16:05:05.031358 28510 consensus_queue.cc:260] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:05.032202 28510 ts_tablet_manager.cc:1428] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260929 16:05:05.032449 28512 tablet_bootstrap.cc:654] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:05.033170 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.032489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.131:49211)
0929 16:05:05.032908 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.033004 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298}
I20260929 16:05:05.034379 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.033889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.131:53069)
0929 16:05:05.034087 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.034179 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 16:05:05.034651 28397 heartbeater.cc:503] Master 127.20.121.188:41459 was elected leader, sending a full tablet report...
I20260929 16:05:05.037113 28512 tablet_bootstrap.cc:492] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: No bootstrap required, opened a new log
I20260929 16:05:05.037305 28512 ts_tablet_manager.cc:1397] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 16:05:05.037971 28512 raft_consensus.cc:348] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:05.038244 28512 raft_consensus.cc:374] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:05.038345 28512 raft_consensus.cc:729] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1600f1ed9e3c4cda9feac1f01ae5d3e0, State: Initialized, Role: FOLLOWER
I20260929 16:05:05.038605 28512 consensus_queue.cc:260] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:05.039922 28512 ts_tablet_manager.cc:1428] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 16:05:05.040271 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.039852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.129:49129)
0929 16:05:05.040018 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.040120 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 16:05:05.041224 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.040829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.129:50629)
0929 16:05:05.040977 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.041076 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 16:05:05.041520 28106 heartbeater.cc:503] Master 127.20.121.188:41459 was elected leader, sending a full tablet report...
I20260929 16:05:05.064965 28513 raft_consensus.cc:482] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:05:05.065231 28513 raft_consensus.cc:504] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:05.066861 28513 leader_election.cc:288] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5188495bc43c49e48ebdf870aa0b5d80 (127.20.121.130:37735), 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627)
I20260929 16:05:05.073745 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.067306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:54239 (local address 127.20.121.130:37735)
0929 16:05:05.067506 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:05.067514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:05.067603 (+ 89us) server_negotiation.cc:407] Connection header received
0929 16:05:05.067847 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:05.067853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:05.067929 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:05.068054 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:05.068630 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.069822 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.071861 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.072786 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.072984 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.073063 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:05.073335 (+ 272us) server_negotiation.cc:299] Negotiation successful
0929 16:05:05.073471 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 16:05:05.073822 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.067183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:37735 (local address 127.20.121.131:54239)
0929 16:05:05.067472 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.067498 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:05.067689 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:05.068141 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:05.068146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:05.068194 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:05.068439 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:05.068453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.069964 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.069969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:05.071672 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:05.071685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.072978 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.072984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:05.073171 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.073196 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 16:05:05.073308 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 16:05:05.073527 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":49}
I20260929 16:05:05.074575 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.067314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:50255 (local address 127.20.121.129:44627)
0929 16:05:05.067482 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:05.067488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:05.068104 (+ 616us) server_negotiation.cc:407] Connection header received
0929 16:05:05.068266 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:05.068282 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:05.068363 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:05.068465 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:05.069294 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.070447 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.072313 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.073290 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.073486 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.073655 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:05.074081 (+ 426us) server_negotiation.cc:299] Negotiation successful
0929 16:05:05.074291 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 16:05:05.074846 28210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e263cfd1a844b9a413a4245598f960" candidate_uuid: "37e9aca105a7457c8459e7c732a12f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5188495bc43c49e48ebdf870aa0b5d80" is_pre_election: true
I20260929 16:05:05.075318 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.067183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:44627 (local address 127.20.121.131:50255)
0929 16:05:05.067991 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.068012 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:05.068123 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:05.068680 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:05.068690 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:05.068722 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:05.069036 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:05.069049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.070606 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.070612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:05.072140 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:05.072153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.073571 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.073583 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:05.073857 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.073893 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 16:05:05.074755 (+ 862us) client_negotiation.cc:240] Negotiation successful
0929 16:05:05.074920 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":38,"thread_start_us":403,"threads_started":1}
I20260929 16:05:05.075461 28210 raft_consensus.cc:2454] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37e9aca105a7457c8459e7c732a12f20 in term 0.
I20260929 16:05:05.076200 28287 leader_election.cc:302] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e9aca105a7457c8459e7c732a12f20, 5188495bc43c49e48ebdf870aa0b5d80; no voters:
I20260929 16:05:05.076398 28061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e263cfd1a844b9a413a4245598f960" candidate_uuid: "37e9aca105a7457c8459e7c732a12f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" is_pre_election: true
I20260929 16:05:05.076661 28061 raft_consensus.cc:2454] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37e9aca105a7457c8459e7c732a12f20 in term 0.
I20260929 16:05:05.076718 28513 raft_consensus.cc:2781] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:05:05.076825 28513 raft_consensus.cc:482] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:05:05.076920 28513 raft_consensus.cc:3037] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:05.079330 28513 raft_consensus.cc:504] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:05.080024 28513 leader_election.cc:288] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [CANDIDATE]: Term 1 election: Requested vote from peers 5188495bc43c49e48ebdf870aa0b5d80 (127.20.121.130:37735), 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627)
I20260929 16:05:05.080582 28061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e263cfd1a844b9a413a4245598f960" candidate_uuid: "37e9aca105a7457c8459e7c732a12f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0"
I20260929 16:05:05.080850 28061 raft_consensus.cc:3037] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:05.080742 28210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e263cfd1a844b9a413a4245598f960" candidate_uuid: "37e9aca105a7457c8459e7c732a12f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5188495bc43c49e48ebdf870aa0b5d80"
I20260929 16:05:05.081022 28210 raft_consensus.cc:3037] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:05.083232 28210 raft_consensus.cc:2454] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37e9aca105a7457c8459e7c732a12f20 in term 1.
I20260929 16:05:05.083745 28287 leader_election.cc:302] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e9aca105a7457c8459e7c732a12f20, 5188495bc43c49e48ebdf870aa0b5d80; no voters:
I20260929 16:05:05.084064 28513 raft_consensus.cc:2781] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:05.084247 28061 raft_consensus.cc:2454] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37e9aca105a7457c8459e7c732a12f20 in term 1.
W20260929 16:05:05.084503 28109 tablet.cc:2369] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:05:05.085528 28513 raft_consensus.cc:686] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 1 LEADER]: Becoming Leader. State: Replica: 37e9aca105a7457c8459e7c732a12f20, State: Running, Role: LEADER
I20260929 16:05:05.085969 28513 consensus_queue.cc:237] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } }
I20260929 16:05:05.087555 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.086978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.131:57875)
0929 16:05:05.087225 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.087337 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260929 16:05:05.090230 27924 catalog_manager.cc:5495] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37e9aca105a7457c8459e7c732a12f20 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "37e9aca105a7457c8459e7c732a12f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37e9aca105a7457c8459e7c732a12f20" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 33721 } health_report { overall_health: HEALTHY } } }
I20260929 16:05:05.121618 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.115333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40650 (local address 127.20.121.131:33721)
0929 16:05:05.115666 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:05.115671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:05.115691 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:05:05.115785 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:05.115789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:05.115871 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:05.116018 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:05.116499 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.117227 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.118896 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.119420 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.119563 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.120215 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:05.120844 (+ 629us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:05.120984 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:05.121188 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 16:05:05.121358 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260929 16:05:05.126307 28061 raft_consensus.cc:1261] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 1 FOLLOWER]: Refusing update from remote peer 37e9aca105a7457c8459e7c732a12f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:05.126559 28210 raft_consensus.cc:1261] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80 [term 1 FOLLOWER]: Refusing update from remote peer 37e9aca105a7457c8459e7c732a12f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:05.127094 28517 consensus_queue.cc:1035] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1600f1ed9e3c4cda9feac1f01ae5d3e0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:05.127483 28513 consensus_queue.cc:1035] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5188495bc43c49e48ebdf870aa0b5d80" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:05.130920 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.130469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.130:59833)
0929 16:05:05.130626 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.130726 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 16:05:05.131283 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.130847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.129:49259)
0929 16:05:05.131065 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.131153 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260929 16:05:05.135421 28520 mvcc.cc:205] Tried to move back new op lower bound from 7334698619388563456 to 7334698619236851712. Current Snapshot: MvccSnapshot[applied={T|T < 7334698619388563456}]
I20260929 16:05:05.136668 28521 mvcc.cc:205] Tried to move back new op lower bound from 7334698619388563456 to 7334698619236851712. Current Snapshot: MvccSnapshot[applied={T|T < 7334698619388563456}]
I20260929 16:05:05.137422 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.136601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.129:48507)
0929 16:05:05.137028 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.137167 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255}
I20260929 16:05:05.140982 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.139714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.131:48543)
0929 16:05:05.140591 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.140733 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":710}
I20260929 16:05:05.147918 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.147242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.130:55169)
0929 16:05:05.147493 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.147628 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260929 16:05:05.152100 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.150473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.20.121.131:50109)
0929 16:05:05.151682 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.151820 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1046}
I20260929 16:05:05.171607 27925 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:05.171945 27925 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:05.178876 27925 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 30c111a79abb4f979bd085b0ac6c3579: Sending DeleteTablet for 3 replicas of tablet 70e263cfd1a844b9a413a4245598f960
I20260929 16:05:05.179999 28190 tablet_service.cc:1514] Processing DeleteTablet for tablet 70e263cfd1a844b9a413a4245598f960 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:05 UTC) from {username='oryx', principal='oryx/127.20.121.188@KRBTEST.COM'} at 127.0.0.1:47760
I20260929 16:05:05.180323 28041 tablet_service.cc:1514] Processing DeleteTablet for tablet 70e263cfd1a844b9a413a4245598f960 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:05 UTC) from {username='oryx', principal='oryx/127.20.121.188@KRBTEST.COM'} at 127.0.0.1:60630
I20260929 16:05:05.181072 28531 tablet_replica.cc:330] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: stopping tablet replica
I20260929 16:05:05.181217 28530 tablet_replica.cc:330] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80: stopping tablet replica
I20260929 16:05:05.181263 28332 tablet_service.cc:1514] Processing DeleteTablet for tablet 70e263cfd1a844b9a413a4245598f960 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:05 UTC) from {username='oryx', principal='oryx/127.20.121.188@KRBTEST.COM'} at 127.0.0.1:40642
I20260929 16:05:05.181391 28530 raft_consensus.cc:2229] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:05.181560 28531 raft_consensus.cc:2229] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:05.181741 28530 raft_consensus.cc:2258] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:05.182629 28531 raft_consensus.cc:2258] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:05.183629 28532 tablet_replica.cc:330] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20: stopping tablet replica
I20260929 16:05:05.183911 28532 raft_consensus.cc:2229] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:05:05.183939 28530 ts_tablet_manager.cc:1905] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:05.184440 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 27962
I20260929 16:05:05.184670 28532 raft_consensus.cc:2258] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:05.185926 28532 ts_tablet_manager.cc:1905] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:05.188360 28531 ts_tablet_manager.cc:1905] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:05.194597 28532 ts_tablet_manager.cc:1918] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:05.194883 28532 log.cc:1196] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/70e263cfd1a844b9a413a4245598f960
I20260929 16:05:05.195523 28532 ts_tablet_manager.cc:1939] T 70e263cfd1a844b9a413a4245598f960 P 37e9aca105a7457c8459e7c732a12f20: Deleting consensus metadata
I20260929 16:05:05.195897 28531 ts_tablet_manager.cc:1918] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:05.196117 28531 log.cc:1196] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/70e263cfd1a844b9a413a4245598f960
I20260929 16:05:05.196646 28531 ts_tablet_manager.cc:1939] T 70e263cfd1a844b9a413a4245598f960 P 1600f1ed9e3c4cda9feac1f01ae5d3e0: Deleting consensus metadata
I20260929 16:05:05.197503 27910 catalog_manager.cc:4841] TS 37e9aca105a7457c8459e7c732a12f20 (127.20.121.131:33721): tablet 70e263cfd1a844b9a413a4245598f960 (table test-table [id=9492079194134c18b8afbe792540258a]) successfully deleted
I20260929 16:05:05.197856 27909 catalog_manager.cc:4841] TS 1600f1ed9e3c4cda9feac1f01ae5d3e0 (127.20.121.129:44627): tablet 70e263cfd1a844b9a413a4245598f960 (table test-table [id=9492079194134c18b8afbe792540258a]) successfully deleted
I20260929 16:05:05.202186 28530 ts_tablet_manager.cc:1918] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:05.202370 28530 log.cc:1196] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/70e263cfd1a844b9a413a4245598f960
I20260929 16:05:05.202862 28530 ts_tablet_manager.cc:1939] T 70e263cfd1a844b9a413a4245598f960 P 5188495bc43c49e48ebdf870aa0b5d80: Deleting consensus metadata
I20260929 16:05:05.204101 27911 catalog_manager.cc:4841] TS 5188495bc43c49e48ebdf870aa0b5d80 (127.20.121.130:37735): tablet 70e263cfd1a844b9a413a4245598f960 (table test-table [id=9492079194134c18b8afbe792540258a]) successfully deleted
I20260929 16:05:05.213878 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.185700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.20.121.129:44627)
0929 16:05:05.185866 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:05.185884 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:05.198569 (+ 12685us) server_negotiation.cc:407] Connection header received
0929 16:05:05.198798 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:05.198803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:05.198889 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:05.199041 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:05.200648 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.201484 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.203106 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.203511 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.203669 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.206655 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:05.206670 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:05.206685 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:05.206726 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:05.209940 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:05.210960 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:05.210971 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:05.210974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:05.211145 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:05.211577 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:05.211584 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:05.211588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:05.212001 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:05.212644 (+ 643us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:05.213109 (+ 465us) server_negotiation.cc:299] Negotiation successful
0929 16:05:05.213399 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 16:05:05.214913 28101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:05.312829 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 27962
I20260929 16:05:05.337414 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 28118
I20260929 16:05:05.354933 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.338325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47774 (local address 127.20.121.130:37735)
0929 16:05:05.338516 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:05.338525 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:05.338547 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:05:05.338737 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:05.338744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:05.338825 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:05.338937 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:05.340436 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.341567 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.343027 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.343730 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.343871 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.347145 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:05.347169 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:05.347192 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:05.347244 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:05.351544 (+ 4300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:05.352460 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:05.352470 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:05.352475 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:05.352636 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:05.353104 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:05.353108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:05.353111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:05.353591 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:05.353905 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:05.354298 (+ 393us) server_negotiation.cc:299] Negotiation successful
0929 16:05:05.354544 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 16:05:05.355786 28250 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:05.504290 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 28118
I20260929 16:05:05.527405 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 28268
I20260929 16:05:05.544272 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.528333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.20.121.131:33721)
0929 16:05:05.528612 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:05.528618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:05.528641 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:05:05.528755 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:05.528760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:05.528864 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:05.529001 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:05.530670 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.531628 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.533018 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:05.533499 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:05.533673 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:05.537083 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:05.537097 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:05.537115 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:05.537160 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:05.540830 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:05.541706 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:05.541715 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:05.541717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:05.541892 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:05.542423 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:05.542429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:05.542440 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:05.542869 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:05.543230 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:05.543643 (+ 413us) server_negotiation.cc:299] Negotiation successful
0929 16:05:05.543907 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260929 16:05:05.545084 28392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:05.596489 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.595659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56828)
0929 16:05:05.596145 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.596269 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":227,"threads_started":1}
I20260929 16:05:05.599645 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:05.598890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:43697 (local address 127.0.0.1:56842)
0929 16:05:05.599078 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:05.599455 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:43697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 16:05:05.685197 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 28268
I20260929 16:05:05.708626 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 27600
I20260929 16:05:05.710134 27768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:05.831279 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 27600
I20260929 16:05:05.854701 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 27701
I20260929 16:05:05.855926 27955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:05.988685 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 27701
2026-09-29T16:05:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11709 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:05:06 dist-test-slave-x07d krb5kdc[28543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:05:06 dist-test-slave-x07d krb5kdc[28543](info): set up 2 sockets
Sep 29 16:05:06 dist-test-slave-x07d krb5kdc[28543](info): commencing operation
krb5kdc: starting...
W20260929 16:05:07.179607 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:05:07 dist-test-slave-x07d krb5kdc[28543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:05:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:05:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.190.keytab.
Entry for principal oryx/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.190.keytab.
I20260929 16:05:07.411048 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:40033
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.190.keytab
--principal=oryx/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:40033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:07.747897 28556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:07.748225 28556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:07.748404 28556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:07.755852 28556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:05:07.755980 28556 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 16:05:07.756021 28556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:07.756148 28556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:07.756239 28556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:05:07.756326 28556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:05:07.767450 28556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:40033
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.190.keytab
--principal=oryx/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:40033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:07.769327 28556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:07.771076 28556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:07.786053 28561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:07.786177 28562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:07.786228 28564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:07.786984 28556 server_base.cc:1034] running on GCE node
I20260929 16:05:07.788026 28556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:07.789635 28556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:07.790925 28556 hybrid_clock.cc:648] HybridClock initialized: now 1790697907790832 us; error 78 us; skew 500 ppm
I20260929 16:05:07.791343 28556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:07 dist-test-slave-x07d krb5kdc[28543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697907, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:07.795055 28556 init.cc:377] Logged in from keytab as oryx/127.20.121.190@KRBTEST.COM (short username oryx)
I20260929 16:05:07.796841 28556 webserver.cc:460] Webserver started at http://127.20.121.190:45833/ using document root <none> and password file <none>
I20260929 16:05:07.797683 28556 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:07.797842 28556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:07.798306 28556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:07.801882 28556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "d45d86ba489343f3b1b5cfe5a264d68c"
format_stamp: "Formatted at 2026-09-29 16:05:07 on dist-test-slave-x07d"
server_key: "8cf6f218e33b514b9370e976144407ae"
server_key_iv: "14607b760650534c1bfb63912a1f0bbc"
server_key_version: "encryptionkey@0"
I20260929 16:05:07.803012 28556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "d45d86ba489343f3b1b5cfe5a264d68c"
format_stamp: "Formatted at 2026-09-29 16:05:07 on dist-test-slave-x07d"
server_key: "8cf6f218e33b514b9370e976144407ae"
server_key_iv: "14607b760650534c1bfb63912a1f0bbc"
server_key_version: "encryptionkey@0"
I20260929 16:05:07.809423 28556 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260929 16:05:07.813403 28571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:07.815231 28556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:05:07.815503 28556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "d45d86ba489343f3b1b5cfe5a264d68c"
format_stamp: "Formatted at 2026-09-29 16:05:07 on dist-test-slave-x07d"
server_key: "8cf6f218e33b514b9370e976144407ae"
server_key_iv: "14607b760650534c1bfb63912a1f0bbc"
server_key_version: "encryptionkey@0"
I20260929 16:05:07.815778 28556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:07.831439 28556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:07.836645 28556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:07.837013 28556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:07.860352 28556 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:40033
I20260929 16:05:07.860354 28632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:40033 every 8 connection(s)
I20260929 16:05:07.861927 28556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:05:07.864681 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 28556
I20260929 16:05:07.864928 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:05:07.865702 20966 external_mini_cluster.cc:1428] Setting key a6dcd832c9117b61b95ac35c3e6e2d84
I20260929 16:05:07.866745 28633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 16:05:07 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:05:07.881964 28633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c: Bootstrap starting.
I20260929 16:05:07.886312 28633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:07.887717 28633 log.cc:824] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:07.891501 28633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c: No bootstrap required, opened a new log
I20260929 16:05:07.892583 28636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:07.867922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.20.121.190:40033)
0929 16:05:07.868874 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:07.868894 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:07.868950 (+ 56us) server_negotiation.cc:407] Connection header received
0929 16:05:07.870821 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:07.870852 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:07.871487 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:07.871993 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:07.873568 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:07.875694 (+ 2126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:07.877222 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:07.878376 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:07.878581 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:07.883157 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:07.883168 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:07.883190 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:07.883242 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:07.887358 (+ 4116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:07.888350 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:07.888363 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:07.888377 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:07.888594 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:07.889165 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:07.889170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:07.889173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:07.889903 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:07.890236 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:07.890669 (+ 433us) server_negotiation.cc:299] Negotiation successful
0929 16:05:07.891162 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":429,"threads_started":1}
I20260929 16:05:07.900638 28633 raft_consensus.cc:348] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45d86ba489343f3b1b5cfe5a264d68c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40033 } }
I20260929 16:05:07.901041 28633 raft_consensus.cc:374] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:07.901175 28633 raft_consensus.cc:729] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d45d86ba489343f3b1b5cfe5a264d68c, State: Initialized, Role: FOLLOWER
I20260929 16:05:07.901928 28633 consensus_queue.cc:260] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45d86ba489343f3b1b5cfe5a264d68c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40033 } }
I20260929 16:05:07.902263 28633 raft_consensus.cc:388] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:05:07.902407 28633 raft_consensus.cc:482] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:05:07.902597 28633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:07.904855 28633 raft_consensus.cc:504] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45d86ba489343f3b1b5cfe5a264d68c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40033 } }
I20260929 16:05:07.905472 28633 leader_election.cc:302] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d45d86ba489343f3b1b5cfe5a264d68c; no voters:
I20260929 16:05:07.906253 28633 leader_election.cc:288] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:05:07.906420 28638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:07.907218 28638 raft_consensus.cc:686] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [term 1 LEADER]: Becoming Leader. State: Replica: d45d86ba489343f3b1b5cfe5a264d68c, State: Running, Role: LEADER
I20260929 16:05:07.907827 28638 consensus_queue.cc:237] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45d86ba489343f3b1b5cfe5a264d68c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40033 } }
I20260929 16:05:07.908591 28633 sys_catalog.cc:564] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:05:07.911099 28639 sys_catalog.cc:455] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d45d86ba489343f3b1b5cfe5a264d68c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45d86ba489343f3b1b5cfe5a264d68c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40033 } } }
I20260929 16:05:07.911546 28640 sys_catalog.cc:455] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d45d86ba489343f3b1b5cfe5a264d68c. Latest consensus state: current_term: 1 leader_uuid: "d45d86ba489343f3b1b5cfe5a264d68c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d45d86ba489343f3b1b5cfe5a264d68c" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 40033 } } }
I20260929 16:05:07.911926 28640 sys_catalog.cc:458] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:07.911931 28639 sys_catalog.cc:458] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:07.912894 28646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:05:07.918238 28646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:05:07.928469 28646 catalog_manager.cc:1347] Generated new cluster ID: 3f9f0e5a05c74699848024697add57f6
I20260929 16:05:07.928599 28646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:05:07.934455 28656 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 16:05:07.936573 28646 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:05:07.938191 28646 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:05:07.946678 28646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c: Generated new TSK 0
I20260929 16:05:07.947685 28646 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab.
Entry for principal oryx/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab.
I20260929 16:05:08.011373 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:40033
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab
--principal=oryx/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:08.357060 28659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:08.357363 28659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:08.357564 28659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:08.365185 28659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:08.365326 28659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:08.365455 28659 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 16:05:08.365547 28659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:05:08.375787 28659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab
--principal=oryx/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.121.190:40033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:08.377054 28659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:08.378463 28659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:08.393039 28665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:08.393080 28664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:08.393795 28667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:08.394191 28659 server_base.cc:1034] running on GCE node
I20260929 16:05:08.394912 28659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:08.396005 28659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:08.397251 28659 hybrid_clock.cc:648] HybridClock initialized: now 1790697908397188 us; error 44 us; skew 500 ppm
I20260929 16:05:08.397727 28659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:08 dist-test-slave-x07d krb5kdc[28543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697908, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:08.401497 28659 init.cc:377] Logged in from keytab as oryx/127.20.121.129@KRBTEST.COM (short username oryx)
I20260929 16:05:08.403373 28659 webserver.cc:460] Webserver started at http://127.20.121.129:45487/ using document root <none> and password file <none>
I20260929 16:05:08.404168 28659 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:08.404264 28659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:08.404656 28659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:08.407891 28659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c"
format_stamp: "Formatted at 2026-09-29 16:05:08 on dist-test-slave-x07d"
server_key: "e1be507f41d1ae7ac600474fd538d171"
server_key_iv: "0b6fc369e8972d6930e5fb20bad1d816"
server_key_version: "encryptionkey@0"
I20260929 16:05:08.408916 28659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c"
format_stamp: "Formatted at 2026-09-29 16:05:08 on dist-test-slave-x07d"
server_key: "e1be507f41d1ae7ac600474fd538d171"
server_key_iv: "0b6fc369e8972d6930e5fb20bad1d816"
server_key_version: "encryptionkey@0"
I20260929 16:05:08.415328 28659 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260929 16:05:08.419400 28674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:08.421198 28659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:05:08.421413 28659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c"
format_stamp: "Formatted at 2026-09-29 16:05:08 on dist-test-slave-x07d"
server_key: "e1be507f41d1ae7ac600474fd538d171"
server_key_iv: "0b6fc369e8972d6930e5fb20bad1d816"
server_key_version: "encryptionkey@0"
I20260929 16:05:08.421656 28659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:08.441408 28659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:08.446616 28659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:08.446983 28659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:08.450253 28659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:08.450397 28659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:08.450544 28659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:08.450629 28659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:05:08 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697908, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:05:08.477150 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:08.450514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:38575 (local address 127.20.121.190:40033)
0929 16:05:08.451045 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:08.451052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:08.452494 (+ 1442us) server_negotiation.cc:407] Connection header received
0929 16:05:08.456098 (+ 3604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:08.456104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:08.456203 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:08.456919 (+ 716us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:08.458960 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:08.460040 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:08.462857 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:08.463656 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:08.463853 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:08.468161 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:08.468172 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:08.468180 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:08.468244 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:08.472110 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:08.473605 (+ 1495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:08.473614 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:08.473619 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:08.473770 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:08.474524 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:08.474528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:08.474531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:08.474907 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:08.475037 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:08.476348 (+ 1311us) server_negotiation.cc:299] Negotiation successful
0929 16:05:08.476625 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":290,"threads_started":1}
I20260929 16:05:08.479293 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:08.451314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40033 (local address 127.20.121.129:38575)
0929 16:05:08.452272 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:08.452330 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:08.455508 (+ 3178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:08.456708 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:08.456729 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:08.457614 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:08.458652 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:08.458692 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:08.460227 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:08.460231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:08.462645 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:08.462669 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:08.464045 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:08.464053 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:08.464294 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:08.465063 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:08.465088 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:08.467814 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:08.472350 (+ 4536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:08.472371 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:08.472401 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:08.473422 (+ 1021us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:08.473905 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:08.473914 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:08.473921 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:08.474349 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:08.475059 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:08.475073 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:08.475765 (+ 692us) client_negotiation.cc:769] Sending connection context
0929 16:05:08.476181 (+ 416us) client_negotiation.cc:240] Negotiation successful
0929 16:05:08.476975 (+ 794us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":375,"threads_started":1}
I20260929 16:05:08.502552 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:08.486589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:60485 (local address 127.20.121.190:40033)
0929 16:05:08.486773 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:08.486778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:08.487614 (+ 836us) server_negotiation.cc:407] Connection header received
0929 16:05:08.487843 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:08.487848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:08.487912 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:08.488023 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:08.489032 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:08.489747 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:08.492085 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:08.492753 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:08.492902 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:08.496048 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:08.496057 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:08.496064 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:08.496124 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:08.499698 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:08.500530 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:08.500536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:08.500541 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:08.500674 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:08.501080 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:08.501083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:08.501085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:08.501360 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:08.501532 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:08.501929 (+ 397us) server_negotiation.cc:299] Negotiation successful
0929 16:05:08.502184 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 16:05:08.502769 28748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:08.486649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40033 (local address 127.20.121.129:60485)
0929 16:05:08.487480 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:08.487498 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:08.487695 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:08.488048 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:08.488052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:08.488552 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:08.488863 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:08.488873 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:08.489910 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:08.489947 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:08.491841 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:08.491873 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:08.492996 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:08.493003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:08.493189 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:08.494060 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:08.494085 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:08.495339 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:08.499920 (+ 4581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:08.499927 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:08.499932 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:08.500389 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:08.500793 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:08.500799 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:08.500803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:08.500941 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:08.501487 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:08.501491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:08.501678 (+ 187us) client_negotiation.cc:769] Sending connection context
0929 16:05:08.501825 (+ 147us) client_negotiation.cc:240] Negotiation successful
0929 16:05:08.502217 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":536,"threads_started":1}
I20260929 16:05:08.510524 28659 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:40909
I20260929 16:05:08.510553 28794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:40909 every 8 connection(s)
I20260929 16:05:08.511896 28659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:05:08.514406 28795 heartbeater.cc:347] Connected to a master server at 127.20.121.190:40033
I20260929 16:05:08.514616 28795 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:08.515236 28795 heartbeater.cc:511] Master 127.20.121.190:40033 requested a full tablet report, sending...
I20260929 16:05:08.517254 28588 ts_manager.cc:194] Registered new tserver with Master: b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909)
I20260929 16:05:08.517701 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 28659
I20260929 16:05:08.517930 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:05:08.518733 20966 external_mini_cluster.cc:1428] Setting key cb947a556bfb8450ec2a6d65ff12fb5b
I20260929 16:05:08.519512 28588 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:38575
WARNING: no policy specified for oryx/127.20.121.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab.
Entry for principal oryx/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab.
I20260929 16:05:08.571463 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:40033
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab
--principal=oryx/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:08.921463 28799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:08.921818 28799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:08.921999 28799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:08.929759 28799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:08.929895 28799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:08.930096 28799 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 16:05:08.930248 28799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:05:08.940784 28799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab
--principal=oryx/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.121.190:40033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:08.942174 28799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:08.943578 28799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:08.957804 28805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:08.957989 28804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:08.957818 28807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:08.958673 28799 server_base.cc:1034] running on GCE node
I20260929 16:05:08.959465 28799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:08.960430 28799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:08.961675 28799 hybrid_clock.cc:648] HybridClock initialized: now 1790697908961565 us; error 101 us; skew 500 ppm
I20260929 16:05:08.962095 28799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:08 dist-test-slave-x07d krb5kdc[28543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697908, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:08.965734 28799 init.cc:377] Logged in from keytab as oryx/127.20.121.130@KRBTEST.COM (short username oryx)
I20260929 16:05:08.968951 28799 webserver.cc:460] Webserver started at http://127.20.121.130:41495/ using document root <none> and password file <none>
I20260929 16:05:08.969760 28799 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:08.969854 28799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:08.970279 28799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:08.973505 28799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "889fd5fa55904271be3b51dcc81a03f3"
format_stamp: "Formatted at 2026-09-29 16:05:08 on dist-test-slave-x07d"
server_key: "5fb4242fd336a913fb4cd2e5b3a86e2e"
server_key_iv: "e99e6d0b9697fc54dce0b59ba11d0697"
server_key_version: "encryptionkey@0"
I20260929 16:05:08.974423 28799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "889fd5fa55904271be3b51dcc81a03f3"
format_stamp: "Formatted at 2026-09-29 16:05:08 on dist-test-slave-x07d"
server_key: "5fb4242fd336a913fb4cd2e5b3a86e2e"
server_key_iv: "e99e6d0b9697fc54dce0b59ba11d0697"
server_key_version: "encryptionkey@0"
I20260929 16:05:08.980051 28799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:05:08.983958 28814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:08.985616 28799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:05:08.985823 28799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "889fd5fa55904271be3b51dcc81a03f3"
format_stamp: "Formatted at 2026-09-29 16:05:08 on dist-test-slave-x07d"
server_key: "5fb4242fd336a913fb4cd2e5b3a86e2e"
server_key_iv: "e99e6d0b9697fc54dce0b59ba11d0697"
server_key_version: "encryptionkey@0"
I20260929 16:05:08.986155 28799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:09.028451 28799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:09.033499 28799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:09.033826 28799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:09.036873 28799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:09.037010 28799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:09.037190 28799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:09.037276 28799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:05:09 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697908, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:05:09.065311 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.037654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:39833 (local address 127.20.121.190:40033)
0929 16:05:09.038284 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:09.038291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:09.039705 (+ 1414us) server_negotiation.cc:407] Connection header received
0929 16:05:09.043214 (+ 3509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:09.043223 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:09.043303 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:09.043453 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:09.045888 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.046956 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.049645 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.050231 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.050373 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.056600 (+ 6227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:09.056611 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:09.056620 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:09.056682 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:09.060753 (+ 4071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:09.062001 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:09.062035 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:09.062042 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:09.062241 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:09.062783 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:09.062791 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:09.062797 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:09.063307 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:09.063526 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:09.064333 (+ 807us) server_negotiation.cc:299] Negotiation successful
0929 16:05:09.064702 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":391,"threads_started":1}
I20260929 16:05:09.066115 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.038479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40033 (local address 127.20.121.130:39833)
0929 16:05:09.039484 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:09.039540 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:09.042767 (+ 3227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:09.043807 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:09.043821 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:09.044786 (+ 965us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:09.045678 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.045701 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.047231 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.047237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.049459 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.049477 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.050594 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.050602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.050966 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.051936 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:09.052002 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:09.056295 (+ 4293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:09.061041 (+ 4746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:09.061054 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:09.061090 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:09.061803 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:09.062408 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:09.062416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:09.062420 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:09.062656 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:09.063494 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:09.063500 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:09.063937 (+ 437us) client_negotiation.cc:769] Sending connection context
0929 16:05:09.064201 (+ 264us) client_negotiation.cc:240] Negotiation successful
0929 16:05:09.064655 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":406,"threads_started":1}
I20260929 16:05:09.086707 28823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.072424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:38601 (local address 127.20.121.190:40033)
0929 16:05:09.072696 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:09.072706 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:09.073264 (+ 558us) server_negotiation.cc:407] Connection header received
0929 16:05:09.073461 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:09.073471 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:09.073588 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:09.073747 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:09.074999 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.075738 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.077137 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.077563 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.077705 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.080416 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:09.080421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:09.080424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:09.080457 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:09.083676 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:09.084376 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:09.084384 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:09.084389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:09.084516 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:09.084899 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:09.084903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:09.084908 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:09.085453 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:09.085651 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:09.086101 (+ 450us) server_negotiation.cc:299] Negotiation successful
0929 16:05:09.086351 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 16:05:09.086853 28896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.072535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40033 (local address 127.20.121.130:38601)
0929 16:05:09.073143 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:09.073163 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:09.073317 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:09.073757 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:09.073761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:09.074317 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:09.074818 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.074831 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.075871 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.075879 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.076982 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.076994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.077749 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.077756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.077927 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.078759 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:09.078776 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:09.080135 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:09.083820 (+ 3685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:09.083826 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:09.083830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:09.084243 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:09.084634 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:09.084639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:09.084643 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:09.084786 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:09.085610 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:09.085615 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:09.085826 (+ 211us) client_negotiation.cc:769] Sending connection context
0929 16:05:09.085985 (+ 159us) client_negotiation.cc:240] Negotiation successful
0929 16:05:09.086351 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":333,"threads_started":1}
I20260929 16:05:09.097452 28799 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:39709
I20260929 16:05:09.097472 28934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:39709 every 8 connection(s)
I20260929 16:05:09.098877 28799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:05:09.101109 28935 heartbeater.cc:347] Connected to a master server at 127.20.121.190:40033
I20260929 16:05:09.101369 28935 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:09.102164 28935 heartbeater.cc:511] Master 127.20.121.190:40033 requested a full tablet report, sending...
I20260929 16:05:09.103821 28587 ts_manager.cc:194] Registered new tserver with Master: 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709)
I20260929 16:05:09.104672 28587 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:39833
I20260929 16:05:09.106971 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 28799
I20260929 16:05:09.107182 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:05:09.107784 20966 external_mini_cluster.cc:1428] Setting key 759e0e05f91c8339d166f8cf99824404
WARNING: no policy specified for oryx/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab.
Entry for principal oryx/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab.
I20260929 16:05:09.162537 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:40033
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab
--principal=oryx/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:09.509686 28939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:09.510188 28939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:09.510411 28939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:09.518316 28939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:09.518486 28939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:09.518643 28939 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 16:05:09.518777 28939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:05:09.523456 28795 heartbeater.cc:503] Master 127.20.121.190:40033 was elected leader, sending a full tablet report...
I20260929 16:05:09.529867 28939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab
--principal=oryx/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.121.190:40033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:09.531273 28939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:09.532634 28939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:09.546782 28947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:09.546756 28945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:09.547003 28944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:09.547765 28939 server_base.cc:1034] running on GCE node
I20260929 16:05:09.548553 28939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:09.549714 28939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:09.551059 28939 hybrid_clock.cc:648] HybridClock initialized: now 1790697909550992 us; error 44 us; skew 500 ppm
I20260929 16:05:09.551435 28939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:09 dist-test-slave-x07d krb5kdc[28543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697909, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:09.555176 28939 init.cc:377] Logged in from keytab as oryx/127.20.121.131@KRBTEST.COM (short username oryx)
I20260929 16:05:09.556826 28939 webserver.cc:460] Webserver started at http://127.20.121.131:45121/ using document root <none> and password file <none>
I20260929 16:05:09.557577 28939 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:09.557711 28939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:09.558112 28939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:09.561102 28939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "8bf257fb48fb40518f37cead02f53987"
format_stamp: "Formatted at 2026-09-29 16:05:09 on dist-test-slave-x07d"
server_key: "f0f5de978c838d8f723ab9c80e07c753"
server_key_iv: "5d9ffe1f4fd7e73c8008228cd7c866ad"
server_key_version: "encryptionkey@0"
I20260929 16:05:09.561947 28939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "8bf257fb48fb40518f37cead02f53987"
format_stamp: "Formatted at 2026-09-29 16:05:09 on dist-test-slave-x07d"
server_key: "f0f5de978c838d8f723ab9c80e07c753"
server_key_iv: "5d9ffe1f4fd7e73c8008228cd7c866ad"
server_key_version: "encryptionkey@0"
I20260929 16:05:09.567504 28939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 16:05:09.571267 28954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:09.572932 28939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260929 16:05:09.573130 28939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "8bf257fb48fb40518f37cead02f53987"
format_stamp: "Formatted at 2026-09-29 16:05:09 on dist-test-slave-x07d"
server_key: "f0f5de978c838d8f723ab9c80e07c753"
server_key_iv: "5d9ffe1f4fd7e73c8008228cd7c866ad"
server_key_version: "encryptionkey@0"
I20260929 16:05:09.573369 28939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:09.588800 28939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:09.593667 28939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:09.593981 28939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:09.596882 28939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:09.596995 28939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:09.597113 28939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:09.597185 28939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:05:09 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697909, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:05:09.624518 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.597658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:48395 (local address 127.20.121.190:40033)
0929 16:05:09.598135 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:09.598139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:09.599492 (+ 1353us) server_negotiation.cc:407] Connection header received
0929 16:05:09.602281 (+ 2789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:09.602287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:09.602390 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:09.602508 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:09.605664 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.606819 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.609525 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.610199 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.610384 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.616121 (+ 5737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:09.616132 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:09.616141 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:09.616198 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:09.620822 (+ 4624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:09.621830 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:09.621836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:09.621842 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:09.622061 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:09.622698 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:09.622706 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:09.622710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:09.623004 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:09.623168 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:09.623910 (+ 742us) server_negotiation.cc:299] Negotiation successful
0929 16:05:09.624132 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":233,"threads_started":1}
I20260929 16:05:09.625725 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.598404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40033 (local address 127.20.121.131:48395)
0929 16:05:09.599325 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:09.599365 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:09.601748 (+ 2383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:09.602878 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:09.602895 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:09.604036 (+ 1141us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:09.605304 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.605358 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.607021 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.607036 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.609298 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.609320 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.610525 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.610534 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.610911 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.612058 (+ 1147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:09.612082 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:09.615712 (+ 3630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:09.621027 (+ 5315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:09.621039 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:09.621061 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:09.621683 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:09.622229 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:09.622236 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:09.622244 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:09.622565 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:09.623153 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:09.623162 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:09.623553 (+ 391us) client_negotiation.cc:769] Sending connection context
0929 16:05:09.623838 (+ 285us) client_negotiation.cc:240] Negotiation successful
0929 16:05:09.624403 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":350,"threads_started":1}
I20260929 16:05:09.649645 28939 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:40249
I20260929 16:05:09.649740 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.633690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:60611 (local address 127.20.121.190:40033)
0929 16:05:09.633860 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:09.633864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:09.634839 (+ 975us) server_negotiation.cc:407] Connection header received
0929 16:05:09.635151 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:09.635160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:09.635285 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:09.635401 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:09.637255 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.638114 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.640429 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.640900 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.641059 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.643135 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:09.643141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:09.643145 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:09.643178 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:09.646378 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:09.647447 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:09.647454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:09.647458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:09.647617 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:09.648036 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:09.648044 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:09.648047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:09.648354 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:09.648532 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:09.649038 (+ 506us) server_negotiation.cc:299] Negotiation successful
0929 16:05:09.649307 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 16:05:09.650043 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.633774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40033 (local address 127.20.121.131:60611)
0929 16:05:09.634690 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:09.634717 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:09.634983 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:09.635647 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:09.635651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:09.636426 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:09.637082 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.637096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.638277 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.638283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.640255 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.640270 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.641120 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.641127 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.641301 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.642124 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:09.642143 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:09.642920 (+ 777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:09.646581 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:09.646589 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:09.646594 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:09.647299 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:09.647741 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:09.647747 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:09.647752 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:09.647911 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:09.648486 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:09.648492 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:09.648747 (+ 255us) client_negotiation.cc:769] Sending connection context
0929 16:05:09.648935 (+ 188us) client_negotiation.cc:240] Negotiation successful
0929 16:05:09.649336 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":486,"threads_started":1}
I20260929 16:05:09.649684 29074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:40249 every 8 connection(s)
I20260929 16:05:09.651542 28939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:05:09.654261 29075 heartbeater.cc:347] Connected to a master server at 127.20.121.190:40033
I20260929 16:05:09.654567 29075 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:09.655301 29075 heartbeater.cc:511] Master 127.20.121.190:40033 requested a full tablet report, sending...
I20260929 16:05:09.656836 28587 ts_manager.cc:194] Registered new tserver with Master: 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249)
I20260929 16:05:09.657531 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 28939
I20260929 16:05:09.657732 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:05:09.658089 28587 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:48395
I20260929 16:05:09.658325 20966 external_mini_cluster.cc:1428] Setting key dadff4bda6a9a7a5581093e2242ded79
I20260929 16:05:09.661616 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 16:05:09 dist-test-slave-x07d krb5kdc[28543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 16:05:09 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:09 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
W20260929 16:05:09.717393 28963 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:09.705317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46832 (local address 127.20.121.190:40033)
0929 16:05:09.705628 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:09.705635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:09.705851 (+ 216us) server_negotiation.cc:407] Connection header received
0929 16:05:09.706241 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:09.706246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:09.706319 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:09.706435 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:09.707882 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.708753 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.710503 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.710987 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.711101 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.717037 (+ 5936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46832: BlockingRecv error: recv got EOF from 127.0.0.1:46832 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 29 16:05:09 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697909, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:05:09.736213 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.720675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.20.121.190:40033)
0929 16:05:09.720884 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:09.720893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:09.721289 (+ 396us) server_negotiation.cc:407] Connection header received
0929 16:05:09.721574 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:09.721578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:09.721643 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:09.721751 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:09.723068 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.723793 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.725059 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.725501 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.725662 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.730143 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:09.730150 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:09.730154 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:09.730198 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:09.733297 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:09.734160 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:09.734174 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:09.734177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:09.734307 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:09.734724 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:09.734728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:09.734730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:09.735013 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:09.735275 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:09.735628 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 16:05:09.735870 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 16:05:09.740979 28587 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:05:09.748238 28587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:05:09.786521 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.763796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.20.121.129:40909)
0929 16:05:09.778265 (+ 14469us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:09.778271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:09.778293 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:05:09.778421 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:09.778428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:09.778651 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:09.778845 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:09.779743 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.780705 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.783606 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.785105 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.785268 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.785472 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:09.786078 (+ 606us) server_negotiation.cc:299] Negotiation successful
0929 16:05:09.786239 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14316,"thread_start_us":14114,"threads_started":1}
I20260929 16:05:09.786552 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.761671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40249 (local address 127.0.0.1:44762)
0929 16:05:09.762607 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:09.762651 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:09.762831 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:09.778177 (+ 15346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:09.778184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:09.778231 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:09.778714 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.778728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.781325 (+ 2597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.781330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.783666 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.783676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.785336 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.785342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.785615 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.785678 (+ 63us) client_negotiation.cc:769] Sending connection context
0929 16:05:09.785800 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 16:05:09.786055 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":69,"thread_start_us":732,"threads_started":1}
I20260929 16:05:09.786559 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.763479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40909 (local address 127.0.0.1:46954)
0929 16:05:09.777823 (+ 14344us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:09.777841 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:09.777941 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:09.779057 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:09.779061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:09.779087 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:09.779528 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.779539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.780844 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.780851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.783437 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.783452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.785662 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.785669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.785824 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.785858 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 16:05:09.786038 (+ 180us) client_negotiation.cc:240] Negotiation successful
0929 16:05:09.786170 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14197,"thread_start_us":13827,"threads_started":1}
I20260929 16:05:09.787287 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.762225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39709 (local address 127.0.0.1:47546)
0929 16:05:09.777180 (+ 14955us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:09.777204 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:09.777627 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:09.778676 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:09.778685 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:09.778709 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:09.779237 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.779255 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.781963 (+ 2708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.781968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.784270 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:09.784286 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.786440 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.786448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:09.786655 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.786683 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 16:05:09.786773 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 16:05:09.786911 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14778,"mutex_wait_us":22,"thread_start_us":15304,"threads_started":1}
I20260929 16:05:09.787664 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.761888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.20.121.131:40249)
0929 16:05:09.776243 (+ 14355us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:09.776259 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:09.776318 (+ 59us) server_negotiation.cc:407] Connection header received
0929 16:05:09.776499 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:09.776514 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:09.776893 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:09.779013 (+ 2120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:09.779394 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.781183 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.783814 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.785102 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.786931 (+ 1829us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.787097 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:09.787165 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 16:05:09.787343 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14065,"thread_start_us":15348,"threads_started":1}
I20260929 16:05:09.787961 29096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:09.762433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.20.121.130:39709)
0929 16:05:09.762963 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:09.762972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:09.777313 (+ 14341us) server_negotiation.cc:407] Connection header received
0929 16:05:09.777915 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:09.777925 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:09.778204 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:09.778412 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:09.779462 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.781185 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.784455 (+ 3270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:09.785922 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:09.786117 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:09.786291 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:09.787545 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0929 16:05:09.787720 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":260,"threads_started":1}
I20260929 16:05:09.791412 28865 tablet_service.cc:1467] Processing CreateTablet for tablet 38a0c1b512aa4ede83f293d7e8cd02bd (DEFAULT_TABLE table=test-table [id=142d725ec5444027954f8bebd3923903]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:09.791412 28725 tablet_service.cc:1467] Processing CreateTablet for tablet 38a0c1b512aa4ede83f293d7e8cd02bd (DEFAULT_TABLE table=test-table [id=142d725ec5444027954f8bebd3923903]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:09.791430 29005 tablet_service.cc:1467] Processing CreateTablet for tablet 38a0c1b512aa4ede83f293d7e8cd02bd (DEFAULT_TABLE table=test-table [id=142d725ec5444027954f8bebd3923903]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:09.792829 28865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38a0c1b512aa4ede83f293d7e8cd02bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:09.792815 28725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38a0c1b512aa4ede83f293d7e8cd02bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:09.792870 29005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38a0c1b512aa4ede83f293d7e8cd02bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:09.805384 29099 tablet_bootstrap.cc:492] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: Bootstrap starting.
I20260929 16:05:09.807605 29101 tablet_bootstrap.cc:492] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: Bootstrap starting.
I20260929 16:05:09.809475 29100 tablet_bootstrap.cc:492] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: Bootstrap starting.
I20260929 16:05:09.810321 29099 tablet_bootstrap.cc:654] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:09.812251 29099 log.cc:824] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:09.812463 29101 tablet_bootstrap.cc:654] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:09.813162 29100 tablet_bootstrap.cc:654] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:09.813704 29101 log.cc:824] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:09.814649 29100 log.cc:824] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:09.816704 29099 tablet_bootstrap.cc:492] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: No bootstrap required, opened a new log
I20260929 16:05:09.816875 29101 tablet_bootstrap.cc:492] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: No bootstrap required, opened a new log
I20260929 16:05:09.817030 29099 ts_tablet_manager.cc:1397] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20260929 16:05:09.817160 29101 ts_tablet_manager.cc:1397] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260929 16:05:09.817337 29100 tablet_bootstrap.cc:492] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: No bootstrap required, opened a new log
I20260929 16:05:09.817601 29100 ts_tablet_manager.cc:1397] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 16:05:09.825107 29099 raft_consensus.cc:348] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } }
I20260929 16:05:09.825618 29099 raft_consensus.cc:374] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:09.825716 29099 raft_consensus.cc:729] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf257fb48fb40518f37cead02f53987, State: Initialized, Role: FOLLOWER
I20260929 16:05:09.826378 29099 consensus_queue.cc:260] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } }
I20260929 16:05:09.827615 29075 heartbeater.cc:503] Master 127.20.121.190:40033 was elected leader, sending a full tablet report...
I20260929 16:05:09.828394 29099 ts_tablet_manager.cc:1428] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: Time spent starting tablet: real 0.011s user 0.000s sys 0.012s
I20260929 16:05:09.828385 29100 raft_consensus.cc:348] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } }
I20260929 16:05:09.828855 29100 raft_consensus.cc:374] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:09.828960 29100 raft_consensus.cc:729] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889fd5fa55904271be3b51dcc81a03f3, State: Initialized, Role: FOLLOWER
I20260929 16:05:09.829602 29100 consensus_queue.cc:260] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } }
I20260929 16:05:09.829144 29101 raft_consensus.cc:348] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } }
I20260929 16:05:09.830397 29101 raft_consensus.cc:374] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:09.830513 29101 raft_consensus.cc:729] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b1f07b4e7b4ef48427a2dbbc76987c, State: Initialized, Role: FOLLOWER
I20260929 16:05:09.830890 28935 heartbeater.cc:503] Master 127.20.121.190:40033 was elected leader, sending a full tablet report...
I20260929 16:05:09.831090 29101 consensus_queue.cc:260] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } }
I20260929 16:05:09.831511 29100 ts_tablet_manager.cc:1428] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260929 16:05:09.833051 29101 ts_tablet_manager.cc:1428] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260929 16:05:09.851329 28936 tablet.cc:2369] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:05:09.903635 29076 tablet.cc:2369] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:05:10.014768 28796 tablet.cc:2369] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:05:10.069281 29105 raft_consensus.cc:482] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:05:10.069646 29105 raft_consensus.cc:504] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } }
I20260929 16:05:10.071563 29105 leader_election.cc:288] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909), 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709)
I20260929 16:05:10.072561 29107 raft_consensus.cc:482] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:05:10.072844 29107 raft_consensus.cc:504] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } }
I20260929 16:05:10.079198 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.071900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39709 (local address 127.20.121.131:47339)
0929 16:05:10.072162 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:10.072184 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:10.072318 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:10.072792 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:10.072808 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:10.072837 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:10.073113 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.073125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.074531 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.074536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.076569 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.076590 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.078377 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.078383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.078616 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.078650 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 16:05:10.078738 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 16:05:10.078878 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260929 16:05:10.079939 29096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.072051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:47339 (local address 127.20.121.130:39709)
0929 16:05:10.072301 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:10.072310 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:10.072340 (+ 30us) server_negotiation.cc:407] Connection header received
0929 16:05:10.072439 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:10.072447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:10.072545 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:10.072649 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:10.073304 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.074371 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.076788 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.077970 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.078156 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.078233 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:10.079542 (+ 1309us) server_negotiation.cc:299] Negotiation successful
0929 16:05:10.079696 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 16:05:10.079939 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.071994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:35681 (local address 127.20.121.129:40909)
0929 16:05:10.072266 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:10.072272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:10.072955 (+ 683us) server_negotiation.cc:407] Connection header received
0929 16:05:10.073154 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:10.073162 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:10.073246 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:10.073360 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:10.074033 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.074948 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.077320 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.078635 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.078764 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.078827 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:10.079546 (+ 719us) server_negotiation.cc:299] Negotiation successful
0929 16:05:10.079694 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260929 16:05:10.080775 29108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.071948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40909 (local address 127.20.121.131:35681)
0929 16:05:10.072831 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:10.072849 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:10.072980 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:10.073374 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:10.073379 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:10.073402 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:10.073841 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.073852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.075135 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.075140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.077150 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.077163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.079048 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.079055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.079228 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.079269 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 16:05:10.080311 (+ 1042us) client_negotiation.cc:240] Negotiation successful
0929 16:05:10.080456 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":611,"threads_started":1}
I20260929 16:05:10.080953 28885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38a0c1b512aa4ede83f293d7e8cd02bd" candidate_uuid: "8bf257fb48fb40518f37cead02f53987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889fd5fa55904271be3b51dcc81a03f3" is_pre_election: true
I20260929 16:05:10.081323 29107 leader_election.cc:288] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249), 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709)
I20260929 16:05:10.081557 28885 raft_consensus.cc:2454] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf257fb48fb40518f37cead02f53987 in term 0.
I20260929 16:05:10.082312 28958 leader_election.cc:302] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 889fd5fa55904271be3b51dcc81a03f3, 8bf257fb48fb40518f37cead02f53987; no voters:
I20260929 16:05:10.082191 28750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38a0c1b512aa4ede83f293d7e8cd02bd" candidate_uuid: "8bf257fb48fb40518f37cead02f53987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" is_pre_election: true
I20260929 16:05:10.082657 28750 raft_consensus.cc:2454] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf257fb48fb40518f37cead02f53987 in term 0.
I20260929 16:05:10.082860 29105 raft_consensus.cc:2781] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:05:10.083005 29105 raft_consensus.cc:482] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:05:10.083114 29105 raft_consensus.cc:3037] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:10.086611 29105 raft_consensus.cc:504] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } }
I20260929 16:05:10.087357 29105 leader_election.cc:288] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [CANDIDATE]: Term 1 election: Requested vote from peers b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909), 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709)
I20260929 16:05:10.087828 28750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38a0c1b512aa4ede83f293d7e8cd02bd" candidate_uuid: "8bf257fb48fb40518f37cead02f53987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c"
I20260929 16:05:10.087942 28885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38a0c1b512aa4ede83f293d7e8cd02bd" candidate_uuid: "8bf257fb48fb40518f37cead02f53987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889fd5fa55904271be3b51dcc81a03f3"
I20260929 16:05:10.088081 28750 raft_consensus.cc:3037] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:10.088136 28885 raft_consensus.cc:3037] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:10.091670 28885 raft_consensus.cc:2454] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf257fb48fb40518f37cead02f53987 in term 1.
I20260929 16:05:10.091681 28750 raft_consensus.cc:2454] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf257fb48fb40518f37cead02f53987 in term 1.
I20260929 16:05:10.092216 28956 leader_election.cc:302] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 889fd5fa55904271be3b51dcc81a03f3, 8bf257fb48fb40518f37cead02f53987, b5b1f07b4e7b4ef48427a2dbbc76987c; no voters:
I20260929 16:05:10.092661 29096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.075879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:48865 (local address 127.20.121.130:39709)
0929 16:05:10.081679 (+ 5800us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:10.081690 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:10.084433 (+ 2743us) server_negotiation.cc:407] Connection header received
0929 16:05:10.084764 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:10.084772 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:10.084849 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:10.084934 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:10.085578 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.086704 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.089717 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.091193 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.091316 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.091386 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:10.092214 (+ 828us) server_negotiation.cc:299] Negotiation successful
0929 16:05:10.092359 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5674,"thread_start_us":7774,"threads_started":1}
I20260929 16:05:10.092677 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.075707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:55449 (local address 127.20.121.131:40249)
0929 16:05:10.075887 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:10.075893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:10.084831 (+ 8938us) server_negotiation.cc:407] Connection header received
0929 16:05:10.084951 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:10.084962 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:10.085043 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:10.085129 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:10.085768 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.086579 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.089718 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.091056 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.091216 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.091307 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:10.092216 (+ 909us) server_negotiation.cc:299] Negotiation successful
0929 16:05:10.092390 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 16:05:10.093117 29105 raft_consensus.cc:2781] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:10.093426 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.075105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39709 (local address 127.20.121.129:48865)
0929 16:05:10.084332 (+ 9227us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:10.084355 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:10.084554 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:10.084992 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:10.084996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:10.085017 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:10.085412 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.085424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.086830 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.086839 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.089506 (+ 2667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.089524 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.091618 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.091623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.091899 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.092043 (+ 144us) client_negotiation.cc:769] Sending connection context
0929 16:05:10.092836 (+ 793us) client_negotiation.cc:240] Negotiation successful
0929 16:05:10.092994 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9057,"mutex_wait_us":181,"thread_start_us":8878,"threads_started":1}
I20260929 16:05:10.094131 28885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38a0c1b512aa4ede83f293d7e8cd02bd" candidate_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889fd5fa55904271be3b51dcc81a03f3" is_pre_election: true
I20260929 16:05:10.094202 29110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.075279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40249 (local address 127.20.121.129:55449)
0929 16:05:10.084712 (+ 9433us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:10.084730 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:10.084850 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:10.085173 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:10.085180 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:10.085203 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:10.085623 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.085631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.086716 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.086721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.089507 (+ 2786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.089524 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.091584 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.091593 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.091900 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.092062 (+ 162us) client_negotiation.cc:769] Sending connection context
0929 16:05:10.092895 (+ 833us) client_negotiation.cc:240] Negotiation successful
0929 16:05:10.093006 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9304,"mutex_wait_us":477,"thread_start_us":9110,"threads_started":1}
I20260929 16:05:10.094779 28885 raft_consensus.cc:2379] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5b1f07b4e7b4ef48427a2dbbc76987c in current term 1: Already voted for candidate 8bf257fb48fb40518f37cead02f53987 in this term.
I20260929 16:05:10.093873 29105 raft_consensus.cc:686] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf257fb48fb40518f37cead02f53987, State: Running, Role: LEADER
I20260929 16:05:10.095082 29025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38a0c1b512aa4ede83f293d7e8cd02bd" candidate_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf257fb48fb40518f37cead02f53987" is_pre_election: true
I20260929 16:05:10.095857 29105 consensus_queue.cc:237] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } }
I20260929 16:05:10.096284 28676 leader_election.cc:302] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5b1f07b4e7b4ef48427a2dbbc76987c; no voters: 889fd5fa55904271be3b51dcc81a03f3, 8bf257fb48fb40518f37cead02f53987
I20260929 16:05:10.096774 29107 raft_consensus.cc:2726] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 16:05:10.101328 28587 catalog_manager.cc:5495] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf257fb48fb40518f37cead02f53987 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "8bf257fb48fb40518f37cead02f53987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } health_report { overall_health: UNKNOWN } } }
I20260929 16:05:10.141825 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.135787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44772 (local address 127.20.121.131:40249)
0929 16:05:10.135955 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:10.135960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:10.136102 (+ 142us) server_negotiation.cc:407] Connection header received
0929 16:05:10.136204 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:10.136209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:10.136299 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:10.136435 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:10.136793 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.137586 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.139022 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.139524 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.139688 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.140304 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:10.141011 (+ 707us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:10.141128 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:10.141358 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 16:05:10.141442 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 16:05:10.152597 28885 raft_consensus.cc:1261] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf257fb48fb40518f37cead02f53987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:10.152639 28750 raft_consensus.cc:1261] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Refusing update from remote peer 8bf257fb48fb40518f37cead02f53987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:10.153596 29112 consensus_queue.cc:1035] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:10.153892 29105 consensus_queue.cc:1035] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:10.166601 29116 mvcc.cc:205] Tried to move back new op lower bound from 7334698639971905536 to 7334698639757803520. Current Snapshot: MvccSnapshot[applied={T|T < 7334698639971905536}]
I20260929 16:05:10.178548 28588 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:10.178851 28588 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:10.183866 28588 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c: Sending DeleteTablet for 3 replicas of tablet 38a0c1b512aa4ede83f293d7e8cd02bd
I20260929 16:05:10.185369 29005 tablet_service.cc:1514] Processing DeleteTablet for tablet 38a0c1b512aa4ede83f293d7e8cd02bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:10 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:44762
I20260929 16:05:10.185400 28865 tablet_service.cc:1514] Processing DeleteTablet for tablet 38a0c1b512aa4ede83f293d7e8cd02bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:10 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:47546
I20260929 16:05:10.185400 28725 tablet_service.cc:1514] Processing DeleteTablet for tablet 38a0c1b512aa4ede83f293d7e8cd02bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:10 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:46954
I20260929 16:05:10.186508 29125 tablet_replica.cc:330] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: stopping tablet replica
I20260929 16:05:10.187110 29125 raft_consensus.cc:2229] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:10.187757 29125 raft_consensus.cc:2258] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:10.187613 28588 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:05:10.188519 28588 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:05:10.188997 29124 tablet_replica.cc:330] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: stopping tablet replica
I20260929 16:05:10.189587 29124 raft_consensus.cc:2229] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:10.190802 29125 ts_tablet_manager.cc:1905] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:10.199251 29125 ts_tablet_manager.cc:1918] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:10.199462 29125 log.cc:1196] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/38a0c1b512aa4ede83f293d7e8cd02bd
I20260929 16:05:10.200098 29125 ts_tablet_manager.cc:1939] T 38a0c1b512aa4ede83f293d7e8cd02bd P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting consensus metadata
I20260929 16:05:10.201713 28725 tablet_service.cc:1467] Processing CreateTablet for tablet cb1fd074ff7e4e86a00904bd0b08b341 (DEFAULT_TABLE table=test-table [id=5f87f46cfb5449508c0b0ce42e36ea04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:10.202204 29126 tablet_replica.cc:330] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: stopping tablet replica
I20260929 16:05:10.202241 28725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb1fd074ff7e4e86a00904bd0b08b341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:10.202708 29126 raft_consensus.cc:2229] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:05:10.202989 28573 catalog_manager.cc:4841] TS b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909): tablet 38a0c1b512aa4ede83f293d7e8cd02bd (table test-table [id=142d725ec5444027954f8bebd3923903]) successfully deleted
I20260929 16:05:10.203464 29126 raft_consensus.cc:2258] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:10.201834 28865 tablet_service.cc:1467] Processing CreateTablet for tablet cb1fd074ff7e4e86a00904bd0b08b341 (DEFAULT_TABLE table=test-table [id=5f87f46cfb5449508c0b0ce42e36ea04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:10.204046 28865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb1fd074ff7e4e86a00904bd0b08b341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:10.204931 29124 raft_consensus.cc:2258] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:10.207420 29124 ts_tablet_manager.cc:1905] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:10.209529 29004 tablet_service.cc:1467] Processing CreateTablet for tablet cb1fd074ff7e4e86a00904bd0b08b341 (DEFAULT_TABLE table=test-table [id=5f87f46cfb5449508c0b0ce42e36ea04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:10.210193 29004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb1fd074ff7e4e86a00904bd0b08b341. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:10.215998 29124 ts_tablet_manager.cc:1918] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:10.216296 29124 log.cc:1196] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/38a0c1b512aa4ede83f293d7e8cd02bd
I20260929 16:05:10.217051 29124 ts_tablet_manager.cc:1939] T 38a0c1b512aa4ede83f293d7e8cd02bd P 889fd5fa55904271be3b51dcc81a03f3: Deleting consensus metadata
I20260929 16:05:10.217286 29100 tablet_bootstrap.cc:492] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3: Bootstrap starting.
I20260929 16:05:10.219219 29100 tablet_bootstrap.cc:654] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:10.220280 29126 ts_tablet_manager.cc:1905] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:10.220754 28575 catalog_manager.cc:4841] TS 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709): tablet 38a0c1b512aa4ede83f293d7e8cd02bd (table test-table [id=142d725ec5444027954f8bebd3923903]) successfully deleted
I20260929 16:05:10.221835 29101 tablet_bootstrap.cc:492] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c: Bootstrap starting.
I20260929 16:05:10.222882 29100 tablet_bootstrap.cc:492] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3: No bootstrap required, opened a new log
I20260929 16:05:10.223110 29100 ts_tablet_manager.cc:1397] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 16:05:10.223804 29101 tablet_bootstrap.cc:654] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:10.224171 29100 raft_consensus.cc:348] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.224558 29100 raft_consensus.cc:374] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:10.224766 29100 raft_consensus.cc:729] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889fd5fa55904271be3b51dcc81a03f3, State: Initialized, Role: FOLLOWER
I20260929 16:05:10.225203 29099 tablet_bootstrap.cc:492] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987: Bootstrap starting.
I20260929 16:05:10.225208 29100 consensus_queue.cc:260] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.226186 29100 ts_tablet_manager.cc:1428] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 16:05:10.227196 29099 tablet_bootstrap.cc:654] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:10.231945 29126 ts_tablet_manager.cc:1918] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:10.232239 29126 log.cc:1196] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/38a0c1b512aa4ede83f293d7e8cd02bd
I20260929 16:05:10.232996 29126 ts_tablet_manager.cc:1939] T 38a0c1b512aa4ede83f293d7e8cd02bd P 8bf257fb48fb40518f37cead02f53987: Deleting consensus metadata
I20260929 16:05:10.236023 28573 catalog_manager.cc:4841] TS 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249): tablet 38a0c1b512aa4ede83f293d7e8cd02bd (table test-table [id=142d725ec5444027954f8bebd3923903]) successfully deleted
I20260929 16:05:10.237640 29101 tablet_bootstrap.cc:492] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c: No bootstrap required, opened a new log
I20260929 16:05:10.237829 29099 tablet_bootstrap.cc:492] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987: No bootstrap required, opened a new log
I20260929 16:05:10.237861 29101 ts_tablet_manager.cc:1397] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.005s
I20260929 16:05:10.238056 29099 ts_tablet_manager.cc:1397] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260929 16:05:10.238767 29101 raft_consensus.cc:348] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.239048 29101 raft_consensus.cc:374] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:10.239161 29101 raft_consensus.cc:729] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b1f07b4e7b4ef48427a2dbbc76987c, State: Initialized, Role: FOLLOWER
I20260929 16:05:10.239027 29099 raft_consensus.cc:348] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.239289 29099 raft_consensus.cc:374] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:10.239415 29099 raft_consensus.cc:729] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf257fb48fb40518f37cead02f53987, State: Initialized, Role: FOLLOWER
I20260929 16:05:10.239498 29101 consensus_queue.cc:260] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.239761 29099 consensus_queue.cc:260] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.240209 29101 ts_tablet_manager.cc:1428] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260929 16:05:10.241322 29099 ts_tablet_manager.cc:1428] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260929 16:05:10.472675 29107 raft_consensus.cc:482] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:05:10.472972 29107 raft_consensus.cc:504] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.473855 29107 leader_election.cc:288] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709), 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249)
I20260929 16:05:10.474740 28885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb1fd074ff7e4e86a00904bd0b08b341" candidate_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889fd5fa55904271be3b51dcc81a03f3" is_pre_election: true
I20260929 16:05:10.474829 29025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb1fd074ff7e4e86a00904bd0b08b341" candidate_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf257fb48fb40518f37cead02f53987" is_pre_election: true
I20260929 16:05:10.475026 28885 raft_consensus.cc:2454] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5b1f07b4e7b4ef48427a2dbbc76987c in term 0.
I20260929 16:05:10.475188 29025 raft_consensus.cc:2454] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5b1f07b4e7b4ef48427a2dbbc76987c in term 0.
I20260929 16:05:10.475608 28678 leader_election.cc:302] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 889fd5fa55904271be3b51dcc81a03f3, b5b1f07b4e7b4ef48427a2dbbc76987c; no voters:
I20260929 16:05:10.476084 29107 raft_consensus.cc:2781] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:05:10.476261 29107 raft_consensus.cc:482] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:05:10.476354 29107 raft_consensus.cc:3037] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:10.480445 29107 raft_consensus.cc:504] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.481628 29107 leader_election.cc:288] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [CANDIDATE]: Term 1 election: Requested vote from peers 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709), 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249)
I20260929 16:05:10.482188 28885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb1fd074ff7e4e86a00904bd0b08b341" candidate_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889fd5fa55904271be3b51dcc81a03f3"
I20260929 16:05:10.482457 28885 raft_consensus.cc:3037] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:10.482363 29025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb1fd074ff7e4e86a00904bd0b08b341" candidate_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf257fb48fb40518f37cead02f53987"
I20260929 16:05:10.482645 29025 raft_consensus.cc:3037] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:10.485765 28885 raft_consensus.cc:2454] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5b1f07b4e7b4ef48427a2dbbc76987c in term 1.
I20260929 16:05:10.486444 28678 leader_election.cc:302] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 889fd5fa55904271be3b51dcc81a03f3, b5b1f07b4e7b4ef48427a2dbbc76987c; no voters:
I20260929 16:05:10.486830 29025 raft_consensus.cc:2454] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5b1f07b4e7b4ef48427a2dbbc76987c in term 1.
I20260929 16:05:10.486912 29107 raft_consensus.cc:2781] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:10.487931 29107 raft_consensus.cc:686] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 LEADER]: Becoming Leader. State: Replica: b5b1f07b4e7b4ef48427a2dbbc76987c, State: Running, Role: LEADER
I20260929 16:05:10.488781 29107 consensus_queue.cc:237] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.493257 28587 catalog_manager.cc:5495] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c reported cstate change: term changed from 0 to 1, leader changed from <none> to b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129). New cstate: current_term: 1 leader_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } health_report { overall_health: UNKNOWN } } }
Sep 29 16:05:10 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697907, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.190@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:05:10.586468 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.554003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:40033 (local address 127.0.0.1:46848)
0929 16:05:10.555076 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:10.555100 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:10.555309 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:10.555772 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:10.555783 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:10.556873 (+ 1090us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:10.557197 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.557214 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.558942 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.558951 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.560932 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.560955 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.562190 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.562199 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.562456 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.572667 (+ 10211us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:10.572705 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:10.576596 (+ 3891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:10.581866 (+ 5270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:10.581892 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:10.581904 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:10.582930 (+ 1026us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:10.583510 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:10.583518 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:10.583524 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:10.583728 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:10.584960 (+ 1232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:10.584973 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:10.585238 (+ 265us) client_negotiation.cc:769] Sending connection context
0929 16:05:10.585440 (+ 202us) client_negotiation.cc:240] Negotiation successful
0929 16:05:10.585847 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":691,"threads_started":1}
I20260929 16:05:10.586565 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.553307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.20.121.190:40033)
0929 16:05:10.554537 (+ 1230us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:10.554545 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:10.555237 (+ 692us) server_negotiation.cc:407] Connection header received
0929 16:05:10.555483 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:10.555489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:10.555572 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:10.555705 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:10.557429 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.558760 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.561162 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.561917 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.562140 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.576887 (+ 14747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:10.576896 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:10.576901 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:10.576947 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:10.581645 (+ 4698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:10.583131 (+ 1486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:10.583139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:10.583143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:10.583340 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:10.583900 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:10.583907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:10.583912 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:10.584771 (+ 859us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:10.584942 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:10.585696 (+ 754us) server_negotiation.cc:299] Negotiation successful
0929 16:05:10.586096 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":561,"thread_start_us":365,"threads_started":1}
I20260929 16:05:10.593863 28587 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:46848:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 16:05:10.595409 28587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:05:10.605275 29004 tablet_service.cc:1467] Processing CreateTablet for tablet 265ac9e3e4cf4de6845a4cb2253e024b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c75836047cce4afeabe8cb4472a31768]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 16:05:10.605288 28725 tablet_service.cc:1467] Processing CreateTablet for tablet 265ac9e3e4cf4de6845a4cb2253e024b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c75836047cce4afeabe8cb4472a31768]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 16:05:10.605937 29004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265ac9e3e4cf4de6845a4cb2253e024b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:10.605943 28725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265ac9e3e4cf4de6845a4cb2253e024b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:10.610244 28865 tablet_service.cc:1467] Processing CreateTablet for tablet 265ac9e3e4cf4de6845a4cb2253e024b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c75836047cce4afeabe8cb4472a31768]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 16:05:10.610975 28865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265ac9e3e4cf4de6845a4cb2253e024b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:10.615223 29099 tablet_bootstrap.cc:492] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: Bootstrap starting.
I20260929 16:05:10.617535 29099 tablet_bootstrap.cc:654] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:10.629985 29099 tablet_bootstrap.cc:492] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: No bootstrap required, opened a new log
I20260929 16:05:10.630205 29099 ts_tablet_manager.cc:1397] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.003s
I20260929 16:05:10.631006 29099 raft_consensus.cc:348] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.631253 29099 raft_consensus.cc:374] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:10.631328 29099 raft_consensus.cc:729] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf257fb48fb40518f37cead02f53987, State: Initialized, Role: FOLLOWER
I20260929 16:05:10.631582 29099 consensus_queue.cc:260] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.631995 29105 tablet_replica.cc:439] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } }
I20260929 16:05:10.632278 29105 tablet_replica.cc:442] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:05:10.633810 29099 ts_tablet_manager.cc:1428] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260929 16:05:10.638067 29101 tablet_bootstrap.cc:492] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: Bootstrap starting.
I20260929 16:05:10.639200 29100 tablet_bootstrap.cc:492] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: Bootstrap starting.
I20260929 16:05:10.639832 29101 tablet_bootstrap.cc:654] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:10.640962 29100 tablet_bootstrap.cc:654] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:10.653867 29101 tablet_bootstrap.cc:492] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: No bootstrap required, opened a new log
I20260929 16:05:10.654094 29101 ts_tablet_manager.cc:1397] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20260929 16:05:10.654834 29101 raft_consensus.cc:348] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.655090 29101 raft_consensus.cc:374] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:10.655164 29101 raft_consensus.cc:729] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b1f07b4e7b4ef48427a2dbbc76987c, State: Initialized, Role: FOLLOWER
I20260929 16:05:10.655452 29101 consensus_queue.cc:260] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.655869 29107 tablet_replica.cc:439] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } }
I20260929 16:05:10.656216 29107 tablet_replica.cc:442] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:05:10.658876 29101 ts_tablet_manager.cc:1428] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260929 16:05:10.659643 29100 tablet_bootstrap.cc:492] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: No bootstrap required, opened a new log
I20260929 16:05:10.659838 29100 ts_tablet_manager.cc:1397] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.009s
I20260929 16:05:10.660631 29100 raft_consensus.cc:348] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.660898 29100 raft_consensus.cc:374] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:10.661001 29100 raft_consensus.cc:729] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889fd5fa55904271be3b51dcc81a03f3, State: Initialized, Role: FOLLOWER
I20260929 16:05:10.661296 29100 consensus_queue.cc:260] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.661863 29100 ts_tablet_manager.cc:1428] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260929 16:05:10.661787 29106 tablet_replica.cc:439] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } }
I20260929 16:05:10.662168 29106 tablet_replica.cc:442] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:05:10.706319 29105 raft_consensus.cc:482] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:05:10.706573 29105 raft_consensus.cc:504] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.707247 29105 leader_election.cc:288] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909), 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709)
I20260929 16:05:10.707758 28750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265ac9e3e4cf4de6845a4cb2253e024b" candidate_uuid: "8bf257fb48fb40518f37cead02f53987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" is_pre_election: true
I20260929 16:05:10.707782 28885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265ac9e3e4cf4de6845a4cb2253e024b" candidate_uuid: "8bf257fb48fb40518f37cead02f53987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889fd5fa55904271be3b51dcc81a03f3" is_pre_election: true
I20260929 16:05:10.707980 28750 raft_consensus.cc:2454] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf257fb48fb40518f37cead02f53987 in term 0.
I20260929 16:05:10.708122 28885 raft_consensus.cc:2454] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bf257fb48fb40518f37cead02f53987 in term 0.
I20260929 16:05:10.708469 28956 leader_election.cc:302] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bf257fb48fb40518f37cead02f53987, b5b1f07b4e7b4ef48427a2dbbc76987c; no voters:
I20260929 16:05:10.708750 29105 raft_consensus.cc:2781] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:05:10.708847 29105 raft_consensus.cc:482] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:05:10.708954 29105 raft_consensus.cc:3037] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:10.711266 29105 raft_consensus.cc:504] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.711861 29105 leader_election.cc:288] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [CANDIDATE]: Term 1 election: Requested vote from peers b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909), 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709)
I20260929 16:05:10.712455 28885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265ac9e3e4cf4de6845a4cb2253e024b" candidate_uuid: "8bf257fb48fb40518f37cead02f53987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889fd5fa55904271be3b51dcc81a03f3"
I20260929 16:05:10.712481 28750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265ac9e3e4cf4de6845a4cb2253e024b" candidate_uuid: "8bf257fb48fb40518f37cead02f53987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c"
I20260929 16:05:10.712690 28885 raft_consensus.cc:3037] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:10.712684 28750 raft_consensus.cc:3037] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:10.714771 28885 raft_consensus.cc:2454] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf257fb48fb40518f37cead02f53987 in term 1.
I20260929 16:05:10.714787 28750 raft_consensus.cc:2454] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bf257fb48fb40518f37cead02f53987 in term 1.
I20260929 16:05:10.715318 28958 leader_election.cc:302] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 889fd5fa55904271be3b51dcc81a03f3, 8bf257fb48fb40518f37cead02f53987; no voters:
I20260929 16:05:10.715663 29105 raft_consensus.cc:2781] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:10.716120 29105 raft_consensus.cc:686] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf257fb48fb40518f37cead02f53987, State: Running, Role: LEADER
I20260929 16:05:10.716432 29105 consensus_queue.cc:237] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:10.718096 29142 tablet_replica.cc:439] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: TxnStatusTablet state changed. Reason: New leader 8bf257fb48fb40518f37cead02f53987. Latest consensus state: current_term: 1 leader_uuid: "8bf257fb48fb40518f37cead02f53987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } }
I20260929 16:05:10.718429 29142 tablet_replica.cc:442] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: This TxnStatusTablet replica's current role is: LEADER
I20260929 16:05:10.719090 29144 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 16:05:10.719311 29144 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 16:05:10.719995 28587 catalog_manager.cc:5495] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bf257fb48fb40518f37cead02f53987 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "8bf257fb48fb40518f37cead02f53987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } health_report { overall_health: HEALTHY } } }
I20260929 16:05:10.751188 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.744353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44788 (local address 127.20.121.131:40249)
0929 16:05:10.744897 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:10.744901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:10.744919 (+ 18us) server_negotiation.cc:407] Connection header received
0929 16:05:10.745048 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:10.745053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:10.745132 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:10.745220 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:10.745734 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.746628 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.748681 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.749281 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.749515 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.750090 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:10.750271 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:10.750395 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:10.750638 (+ 243us) server_negotiation.cc:299] Negotiation successful
0929 16:05:10.750790 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":297,"threads_started":1}
I20260929 16:05:10.751171 29138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.744257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40249 (local address 127.0.0.1:44788)
0929 16:05:10.744544 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:10.744568 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:10.744731 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:10.745303 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:10.745309 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:10.745324 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:05:10.745558 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.745572 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.746770 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.746778 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.748470 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.748489 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.749471 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.749479 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.749727 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.749819 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:05:10.750428 (+ 609us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:05:10.750475 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 16:05:10.750579 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 16:05:10.750761 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 16:05:10.755963 28885 raft_consensus.cc:1261] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Refusing update from remote peer 8bf257fb48fb40518f37cead02f53987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:10.756105 28750 raft_consensus.cc:1261] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Refusing update from remote peer 8bf257fb48fb40518f37cead02f53987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:10.756824 29142 consensus_queue.cc:1035] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:10.757192 29105 consensus_queue.cc:1035] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:10.767357 29107 tablet_replica.cc:439] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: TxnStatusTablet state changed. Reason: New leader 8bf257fb48fb40518f37cead02f53987. Latest consensus state: current_term: 1 leader_uuid: "8bf257fb48fb40518f37cead02f53987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } }
I20260929 16:05:10.767746 29107 tablet_replica.cc:442] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:05:10.768694 29106 tablet_replica.cc:439] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: TxnStatusTablet state changed. Reason: New leader 8bf257fb48fb40518f37cead02f53987. Latest consensus state: current_term: 1 leader_uuid: "8bf257fb48fb40518f37cead02f53987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } }
I20260929 16:05:10.768960 29106 tablet_replica.cc:442] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:05:10.772464 29148 mvcc.cc:205] Tried to move back new op lower bound from 7334698642449453056 to 7334698642296836096. Current Snapshot: MvccSnapshot[applied={T|T < 7334698642449453056}]
I20260929 16:05:10.774304 29142 tablet_replica.cc:439] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8bf257fb48fb40518f37cead02f53987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } }
I20260929 16:05:10.774726 29142 tablet_replica.cc:442] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: This TxnStatusTablet replica's current role is: LEADER
I20260929 16:05:10.775103 29142 tablet_replica.cc:439] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8bf257fb48fb40518f37cead02f53987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } }
I20260929 16:05:10.775557 29142 tablet_replica.cc:442] T 265ac9e3e4cf4de6845a4cb2253e024b P 8bf257fb48fb40518f37cead02f53987: This TxnStatusTablet replica's current role is: LEADER
I20260929 16:05:10.775449 29106 tablet_replica.cc:439] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8bf257fb48fb40518f37cead02f53987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } }
I20260929 16:05:10.775694 29106 tablet_replica.cc:442] T 265ac9e3e4cf4de6845a4cb2253e024b P 889fd5fa55904271be3b51dcc81a03f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:05:10.780321 29147 mvcc.cc:205] Tried to move back new op lower bound from 7334698642449453056 to 7334698642296836096. Current Snapshot: MvccSnapshot[applied={T|T < 7334698642449453056 or (T in {7334698642449453056})}]
I20260929 16:05:10.793948 29107 tablet_replica.cc:439] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8bf257fb48fb40518f37cead02f53987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } } }
I20260929 16:05:10.794346 29107 tablet_replica.cc:442] T 265ac9e3e4cf4de6845a4cb2253e024b P b5b1f07b4e7b4ef48427a2dbbc76987c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 16:05:10.802076 29157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.781680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.20.121.129:40909)
0929 16:05:10.793713 (+ 12033us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:10.793721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:10.795821 (+ 2100us) server_negotiation.cc:407] Connection header received
0929 16:05:10.796046 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:10.796052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:10.796153 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:10.796363 (+ 210us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:10.796862 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.797667 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.799595 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.799999 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.800099 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.800599 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:10.801208 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:10.801314 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:10.801500 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 16:05:10.801587 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11889,"thread_start_us":11734,"threads_started":1}
I20260929 16:05:10.819386 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.812660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:41323 (local address 127.20.121.131:40249)
0929 16:05:10.812913 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:10.812920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:10.813418 (+ 498us) server_negotiation.cc:407] Connection header received
0929 16:05:10.813571 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:10.813577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:10.813677 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:10.813797 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:10.814484 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.815764 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.817204 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.817946 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.818134 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.818555 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:10.818705 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:10.818817 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:10.818977 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 16:05:10.819068 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260929 16:05:10.819432 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.812606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40249 (local address 127.20.121.129:41323)
0929 16:05:10.813304 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:10.813320 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:10.813440 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:10.813839 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:10.813843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:10.813853 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:05:10.814236 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.814246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.815931 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.815935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.817061 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.817072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.818198 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.818202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.818315 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.818371 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:05:10.818849 (+ 478us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:05:10.818859 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 16:05:10.818924 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 16:05:10.819081 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":377,"threads_started":1}
I20260929 16:05:10.830497 28885 raft_consensus.cc:1261] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Refusing update from remote peer b5b1f07b4e7b4ef48427a2dbbc76987c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:10.830683 29025 raft_consensus.cc:1261] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Refusing update from remote peer b5b1f07b4e7b4ef48427a2dbbc76987c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:10.831205 29130 consensus_queue.cc:1035] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [LEADER]: Connected to new peer: Peer: permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:10.831664 29107 consensus_queue.cc:1035] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:10.875674 29157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.866752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:33527 (local address 127.20.121.129:40909)
0929 16:05:10.866928 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:10.866935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:10.867951 (+ 1016us) server_negotiation.cc:407] Connection header received
0929 16:05:10.868286 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:10.868293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:10.868413 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:10.868483 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:10.869392 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.870634 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.873090 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.873830 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.873936 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.874791 (+ 855us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:10.874940 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:10.875008 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:10.875257 (+ 249us) server_negotiation.cc:299] Negotiation successful
0929 16:05:10.875350 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 16:05:10.876806 29165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:10.866273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40909 (local address 127.20.121.131:33527)
0929 16:05:10.867839 (+ 1566us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:10.867861 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:10.868087 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:10.868720 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:10.868726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:10.868740 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:05:10.869168 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.869182 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.870825 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.870831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.872903 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:10.872920 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:10.874183 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:10.874190 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:10.874459 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:10.874552 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:05:10.875098 (+ 546us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:05:10.875110 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 16:05:10.876123 (+ 1013us) client_negotiation.cc:240] Negotiation successful
0929 16:05:10.876281 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1350,"thread_start_us":2133,"threads_started":1}
I20260929 16:05:10.917210 28587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:10.917482 28587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:10.921049 28587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c: Sending DeleteTablet for 3 replicas of tablet cb1fd074ff7e4e86a00904bd0b08b341
I20260929 16:05:10.922056 28725 tablet_service.cc:1514] Processing DeleteTablet for tablet cb1fd074ff7e4e86a00904bd0b08b341 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:10 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:46954
I20260929 16:05:10.922159 28865 tablet_service.cc:1514] Processing DeleteTablet for tablet cb1fd074ff7e4e86a00904bd0b08b341 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:10 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:47546
I20260929 16:05:10.923293 29004 tablet_service.cc:1514] Processing DeleteTablet for tablet cb1fd074ff7e4e86a00904bd0b08b341 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:10 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:44762
I20260929 16:05:10.935945 29169 tablet_replica.cc:330] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3: stopping tablet replica
I20260929 16:05:10.936203 29169 raft_consensus.cc:2229] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:10.936533 29169 raft_consensus.cc:2258] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:10.937809 29169 ts_tablet_manager.cc:1905] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:10.949038 29169 ts_tablet_manager.cc:1918] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:05:10.949206 29169 log.cc:1196] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/cb1fd074ff7e4e86a00904bd0b08b341
I20260929 16:05:10.949662 29169 ts_tablet_manager.cc:1939] T cb1fd074ff7e4e86a00904bd0b08b341 P 889fd5fa55904271be3b51dcc81a03f3: Deleting consensus metadata
I20260929 16:05:10.950893 28575 catalog_manager.cc:4841] TS 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709): tablet cb1fd074ff7e4e86a00904bd0b08b341 (table test-table [id=5f87f46cfb5449508c0b0ce42e36ea04]) successfully deleted
I20260929 16:05:10.954581 29170 tablet_replica.cc:330] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987: stopping tablet replica
I20260929 16:05:10.954780 29170 raft_consensus.cc:2229] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:10.955076 29170 raft_consensus.cc:2258] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:10.955693 29168 tablet_replica.cc:330] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c: stopping tablet replica
I20260929 16:05:10.955901 29168 raft_consensus.cc:2229] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:05:10.956494 29168 raft_consensus.cc:2258] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:10.957499 29170 ts_tablet_manager.cc:1905] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:10.959553 29168 ts_tablet_manager.cc:1905] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:10.972160 29170 ts_tablet_manager.cc:1918] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:05:10.972311 29170 log.cc:1196] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/cb1fd074ff7e4e86a00904bd0b08b341
I20260929 16:05:10.972820 29170 ts_tablet_manager.cc:1939] T cb1fd074ff7e4e86a00904bd0b08b341 P 8bf257fb48fb40518f37cead02f53987: Deleting consensus metadata
I20260929 16:05:10.977698 28573 catalog_manager.cc:4841] TS 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249): tablet cb1fd074ff7e4e86a00904bd0b08b341 (table test-table [id=5f87f46cfb5449508c0b0ce42e36ea04]) successfully deleted
I20260929 16:05:10.980458 29168 ts_tablet_manager.cc:1918] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:05:10.980618 29168 log.cc:1196] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/cb1fd074ff7e4e86a00904bd0b08b341
I20260929 16:05:10.981146 29168 ts_tablet_manager.cc:1939] T cb1fd074ff7e4e86a00904bd0b08b341 P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting consensus metadata
I20260929 16:05:10.982396 28573 catalog_manager.cc:4841] TS b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909): tablet cb1fd074ff7e4e86a00904bd0b08b341 (table test-table [id=5f87f46cfb5449508c0b0ce42e36ea04]) successfully deleted
Sep 29 16:05:10 dist-test-slave-x07d krb5kdc[28543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 16:05:11 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:11 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
W20260929 16:05:11.028252 29137 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:11.007817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.20.121.190:40033)
0929 16:05:11.008012 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:11.008020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:11.014583 (+ 6563us) server_negotiation.cc:407] Connection header received
0929 16:05:11.014824 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:11.014830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:11.014923 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:11.014994 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:11.016927 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.018151 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.020087 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.020935 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.021139 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.027961 (+ 6822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46852: BlockingRecv error: recv got EOF from 127.0.0.1:46852 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 29 16:05:11 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM
I20260929 16:05:11.059092 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.040034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46854 (local address 127.20.121.190:40033)
0929 16:05:11.040221 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:11.040228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:11.040673 (+ 445us) server_negotiation.cc:407] Connection header received
0929 16:05:11.040856 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:11.040862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:11.040950 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:11.041069 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:11.042707 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.043857 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.045900 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.046629 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.046773 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.051990 (+ 5217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:11.051999 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:11.052004 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:11.052058 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:11.055955 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:11.056856 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:11.056864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:11.056875 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:11.057035 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:11.057467 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:11.057472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:11.057476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:11.057828 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:11.058137 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:11.058449 (+ 312us) server_negotiation.cc:299] Negotiation successful
0929 16:05:11.058708 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 16:05:11.063232 28587 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:05:11.064018 28587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:05:11.072309 28725 tablet_service.cc:1467] Processing CreateTablet for tablet 145bff8c9ada4b78962c5a3b0a74b0d2 (DEFAULT_TABLE table=test-table [id=149440c905a94934b5e7be6c2bb0288b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:11.072448 28865 tablet_service.cc:1467] Processing CreateTablet for tablet 145bff8c9ada4b78962c5a3b0a74b0d2 (DEFAULT_TABLE table=test-table [id=149440c905a94934b5e7be6c2bb0288b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:11.072815 28725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 145bff8c9ada4b78962c5a3b0a74b0d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:11.072888 28865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 145bff8c9ada4b78962c5a3b0a74b0d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:11.077198 29004 tablet_service.cc:1467] Processing CreateTablet for tablet 145bff8c9ada4b78962c5a3b0a74b0d2 (DEFAULT_TABLE table=test-table [id=149440c905a94934b5e7be6c2bb0288b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:11.077720 29004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 145bff8c9ada4b78962c5a3b0a74b0d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:11.080443 29100 tablet_bootstrap.cc:492] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3: Bootstrap starting.
I20260929 16:05:11.082269 29100 tablet_bootstrap.cc:654] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:11.091782 29101 tablet_bootstrap.cc:492] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c: Bootstrap starting.
I20260929 16:05:11.093725 29101 tablet_bootstrap.cc:654] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:11.096652 29099 tablet_bootstrap.cc:492] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987: Bootstrap starting.
I20260929 16:05:11.100000 29099 tablet_bootstrap.cc:654] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:11.111363 29099 tablet_bootstrap.cc:492] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987: No bootstrap required, opened a new log
I20260929 16:05:11.111533 29099 ts_tablet_manager.cc:1397] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260929 16:05:11.112270 29099 raft_consensus.cc:348] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.112512 29099 raft_consensus.cc:374] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:11.112591 29099 raft_consensus.cc:729] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf257fb48fb40518f37cead02f53987, State: Initialized, Role: FOLLOWER
I20260929 16:05:11.112805 29099 consensus_queue.cc:260] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.113407 29099 ts_tablet_manager.cc:1428] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260929 16:05:11.116125 29100 tablet_bootstrap.cc:492] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3: No bootstrap required, opened a new log
I20260929 16:05:11.116314 29100 ts_tablet_manager.cc:1397] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.008s
I20260929 16:05:11.117148 29100 raft_consensus.cc:348] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.117424 29100 raft_consensus.cc:374] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:11.117511 29100 raft_consensus.cc:729] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889fd5fa55904271be3b51dcc81a03f3, State: Initialized, Role: FOLLOWER
I20260929 16:05:11.117766 29100 consensus_queue.cc:260] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.119129 29100 ts_tablet_manager.cc:1428] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260929 16:05:11.126530 29101 tablet_bootstrap.cc:492] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c: No bootstrap required, opened a new log
I20260929 16:05:11.126732 29101 ts_tablet_manager.cc:1397] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c: Time spent bootstrapping tablet: real 0.035s user 0.007s sys 0.014s
I20260929 16:05:11.127575 29101 raft_consensus.cc:348] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.127852 29101 raft_consensus.cc:374] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:11.129737 29101 raft_consensus.cc:729] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b1f07b4e7b4ef48427a2dbbc76987c, State: Initialized, Role: FOLLOWER
I20260929 16:05:11.130178 29101 consensus_queue.cc:260] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.133386 29101 ts_tablet_manager.cc:1428] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260929 16:05:11.242483 29106 raft_consensus.cc:482] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:05:11.242710 29106 raft_consensus.cc:504] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.244293 29106 leader_election.cc:288] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909), 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249)
I20260929 16:05:11.252434 29157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.244693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:43607 (local address 127.20.121.129:40909)
0929 16:05:11.244891 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:11.244896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:11.245334 (+ 438us) server_negotiation.cc:407] Connection header received
0929 16:05:11.245619 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:11.245627 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:11.245741 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:11.245812 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:11.246647 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.247742 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.249936 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.251324 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.251508 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.251585 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:11.251930 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 16:05:11.252090 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 16:05:11.252862 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.245596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:52143 (local address 127.20.121.131:40249)
0929 16:05:11.245761 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:11.245769 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:11.246308 (+ 539us) server_negotiation.cc:407] Connection header received
0929 16:05:11.246595 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:11.246604 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:11.246680 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:11.246745 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:11.247524 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.248627 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.250721 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.251694 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.251838 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.251922 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:11.252430 (+ 508us) server_negotiation.cc:299] Negotiation successful
0929 16:05:11.252590 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 16:05:11.252912 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.245464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40249 (local address 127.20.121.130:52143)
0929 16:05:11.246207 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:11.246233 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:11.246415 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:11.246911 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:11.246917 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:11.246942 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:11.247338 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:11.247349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.248793 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.248798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:11.250580 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:11.250591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.252067 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.252073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:11.252250 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.252283 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 16:05:11.252376 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 16:05:11.252525 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":24,"thread_start_us":443,"threads_started":1}
I20260929 16:05:11.253527 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.244721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40909 (local address 127.20.121.130:43607)
0929 16:05:11.245213 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:11.245230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:11.245423 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:11.246076 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:11.246086 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:11.246115 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:11.246471 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:11.246485 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.247880 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.247887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:11.249746 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:11.249766 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.251556 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.251563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:11.251731 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.251782 (+ 51us) client_negotiation.cc:769] Sending connection context
0929 16:05:11.253106 (+ 1324us) client_negotiation.cc:240] Negotiation successful
0929 16:05:11.253236 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":3,"thread_start_us":289,"threads_started":1}
I20260929 16:05:11.253710 29024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "145bff8c9ada4b78962c5a3b0a74b0d2" candidate_uuid: "889fd5fa55904271be3b51dcc81a03f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf257fb48fb40518f37cead02f53987" is_pre_election: true
I20260929 16:05:11.254004 29024 raft_consensus.cc:2454] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 889fd5fa55904271be3b51dcc81a03f3 in term 0.
I20260929 16:05:11.254336 28750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "145bff8c9ada4b78962c5a3b0a74b0d2" candidate_uuid: "889fd5fa55904271be3b51dcc81a03f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" is_pre_election: true
I20260929 16:05:11.254637 28750 raft_consensus.cc:2454] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 889fd5fa55904271be3b51dcc81a03f3 in term 0.
I20260929 16:05:11.254673 28816 leader_election.cc:302] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 889fd5fa55904271be3b51dcc81a03f3, 8bf257fb48fb40518f37cead02f53987; no voters:
I20260929 16:05:11.255177 29106 raft_consensus.cc:2781] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:05:11.255308 29106 raft_consensus.cc:482] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:05:11.255419 29106 raft_consensus.cc:3037] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:11.258572 29106 raft_consensus.cc:504] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.259315 29106 leader_election.cc:288] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [CANDIDATE]: Term 1 election: Requested vote from peers b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909), 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249)
I20260929 16:05:11.259898 28750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "145bff8c9ada4b78962c5a3b0a74b0d2" candidate_uuid: "889fd5fa55904271be3b51dcc81a03f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c"
I20260929 16:05:11.259961 29024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "145bff8c9ada4b78962c5a3b0a74b0d2" candidate_uuid: "889fd5fa55904271be3b51dcc81a03f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf257fb48fb40518f37cead02f53987"
I20260929 16:05:11.260141 28750 raft_consensus.cc:3037] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:11.260159 29024 raft_consensus.cc:3037] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:11.263383 28750 raft_consensus.cc:2454] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 889fd5fa55904271be3b51dcc81a03f3 in term 1.
I20260929 16:05:11.263379 29024 raft_consensus.cc:2454] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 889fd5fa55904271be3b51dcc81a03f3 in term 1.
I20260929 16:05:11.264014 28816 leader_election.cc:302] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 889fd5fa55904271be3b51dcc81a03f3, 8bf257fb48fb40518f37cead02f53987; no voters:
I20260929 16:05:11.264367 29106 raft_consensus.cc:2781] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:11.264863 29106 raft_consensus.cc:686] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 LEADER]: Becoming Leader. State: Replica: 889fd5fa55904271be3b51dcc81a03f3, State: Running, Role: LEADER
I20260929 16:05:11.265307 29106 consensus_queue.cc:237] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.269512 28588 catalog_manager.cc:5495] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130). New cstate: current_term: 1 leader_uuid: "889fd5fa55904271be3b51dcc81a03f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } health_report { overall_health: UNKNOWN } } }
I20260929 16:05:11.310132 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.304174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.20.121.130:39709)
0929 16:05:11.304734 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:11.304739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:11.304757 (+ 18us) server_negotiation.cc:407] Connection header received
0929 16:05:11.304837 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:11.304841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:11.304911 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:11.305086 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:11.305500 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.306432 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.307748 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.308189 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.308324 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.308833 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:11.309433 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:11.309607 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:11.309656 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 16:05:11.309735 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":312,"threads_started":1}
I20260929 16:05:11.313787 28750 raft_consensus.cc:1261] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Refusing update from remote peer 889fd5fa55904271be3b51dcc81a03f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:11.313836 29024 raft_consensus.cc:1261] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Refusing update from remote peer 889fd5fa55904271be3b51dcc81a03f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:11.314601 29106 consensus_queue.cc:1035] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:11.314944 29188 consensus_queue.cc:1035] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:11.340037 28587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:11.340402 28587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:11.343957 28587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c: Sending DeleteTablet for 3 replicas of tablet 145bff8c9ada4b78962c5a3b0a74b0d2
I20260929 16:05:11.344949 28865 tablet_service.cc:1514] Processing DeleteTablet for tablet 145bff8c9ada4b78962c5a3b0a74b0d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:11 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:47546
I20260929 16:05:11.344977 28725 tablet_service.cc:1514] Processing DeleteTablet for tablet 145bff8c9ada4b78962c5a3b0a74b0d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:11 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:46954
I20260929 16:05:11.345263 29169 tablet_replica.cc:330] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3: stopping tablet replica
I20260929 16:05:11.345346 29168 tablet_replica.cc:330] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c: stopping tablet replica
I20260929 16:05:11.345474 29169 raft_consensus.cc:2229] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:05:11.345562 29168 raft_consensus.cc:2229] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:11.345798 29168 raft_consensus.cc:2258] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:11.346112 29169 raft_consensus.cc:2258] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:11.347213 29168 ts_tablet_manager.cc:1905] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:11.347179 29004 tablet_service.cc:1514] Processing DeleteTablet for tablet 145bff8c9ada4b78962c5a3b0a74b0d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:11 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:44762
I20260929 16:05:11.347464 29170 tablet_replica.cc:330] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987: stopping tablet replica
I20260929 16:05:11.347667 29170 raft_consensus.cc:2229] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:11.347894 29170 raft_consensus.cc:2258] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:11.350107 29170 ts_tablet_manager.cc:1905] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:11.354890 29168 ts_tablet_manager.cc:1918] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:11.355201 29168 log.cc:1196] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/145bff8c9ada4b78962c5a3b0a74b0d2
I20260929 16:05:11.355787 29168 ts_tablet_manager.cc:1939] T 145bff8c9ada4b78962c5a3b0a74b0d2 P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting consensus metadata
I20260929 16:05:11.356902 29170 ts_tablet_manager.cc:1918] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:11.357072 29170 log.cc:1196] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/145bff8c9ada4b78962c5a3b0a74b0d2
I20260929 16:05:11.357479 29170 ts_tablet_manager.cc:1939] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 8bf257fb48fb40518f37cead02f53987: Deleting consensus metadata
I20260929 16:05:11.357959 28573 catalog_manager.cc:4841] TS b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909): tablet 145bff8c9ada4b78962c5a3b0a74b0d2 (table test-table [id=149440c905a94934b5e7be6c2bb0288b]) successfully deleted
I20260929 16:05:11.358750 28587 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 16:05:11.359431 29169 ts_tablet_manager.cc:1905] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 16:05:11.359827 28587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:05:11.366541 29169 ts_tablet_manager.cc:1918] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:11.366736 29169 log.cc:1196] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/145bff8c9ada4b78962c5a3b0a74b0d2
I20260929 16:05:11.367172 29169 ts_tablet_manager.cc:1939] T 145bff8c9ada4b78962c5a3b0a74b0d2 P 889fd5fa55904271be3b51dcc81a03f3: Deleting consensus metadata
I20260929 16:05:11.368876 28865 tablet_service.cc:1467] Processing CreateTablet for tablet 01bc1e9fe6fa405aab2dff32ea58d166 (DEFAULT_TABLE table=test-table [id=d01b10df848948afbb2d082c23cfecb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:11.369272 28725 tablet_service.cc:1467] Processing CreateTablet for tablet 01bc1e9fe6fa405aab2dff32ea58d166 (DEFAULT_TABLE table=test-table [id=d01b10df848948afbb2d082c23cfecb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:11.369397 28865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01bc1e9fe6fa405aab2dff32ea58d166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:11.369721 28725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01bc1e9fe6fa405aab2dff32ea58d166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:11.374558 29004 tablet_service.cc:1467] Processing CreateTablet for tablet 01bc1e9fe6fa405aab2dff32ea58d166 (DEFAULT_TABLE table=test-table [id=d01b10df848948afbb2d082c23cfecb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:11.375063 29004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01bc1e9fe6fa405aab2dff32ea58d166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:11.377008 29100 tablet_bootstrap.cc:492] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3: Bootstrap starting.
I20260929 16:05:11.379302 29100 tablet_bootstrap.cc:654] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:11.385887 29099 tablet_bootstrap.cc:492] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987: Bootstrap starting.
I20260929 16:05:11.387838 29099 tablet_bootstrap.cc:654] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:11.391644 29101 tablet_bootstrap.cc:492] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c: Bootstrap starting.
I20260929 16:05:11.393721 29101 tablet_bootstrap.cc:654] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:11.394824 29100 tablet_bootstrap.cc:492] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3: No bootstrap required, opened a new log
I20260929 16:05:11.395907 29100 ts_tablet_manager.cc:1397] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.000s
I20260929 16:05:11.397149 29100 raft_consensus.cc:348] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.397778 29100 raft_consensus.cc:374] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:11.397925 29100 raft_consensus.cc:729] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889fd5fa55904271be3b51dcc81a03f3, State: Initialized, Role: FOLLOWER
I20260929 16:05:11.398388 29100 consensus_queue.cc:260] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.400777 29100 ts_tablet_manager.cc:1428] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260929 16:05:11.405625 28573 catalog_manager.cc:4841] TS 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249): tablet 145bff8c9ada4b78962c5a3b0a74b0d2 (table test-table [id=149440c905a94934b5e7be6c2bb0288b]) successfully deleted
I20260929 16:05:11.406634 28575 catalog_manager.cc:4841] TS 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709): tablet 145bff8c9ada4b78962c5a3b0a74b0d2 (table test-table [id=149440c905a94934b5e7be6c2bb0288b]) successfully deleted
I20260929 16:05:11.407931 29101 tablet_bootstrap.cc:492] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c: No bootstrap required, opened a new log
I20260929 16:05:11.408193 29101 ts_tablet_manager.cc:1397] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.003s
I20260929 16:05:11.408883 29101 raft_consensus.cc:348] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.409232 29101 raft_consensus.cc:374] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:11.409322 29101 raft_consensus.cc:729] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b1f07b4e7b4ef48427a2dbbc76987c, State: Initialized, Role: FOLLOWER
I20260929 16:05:11.409579 29101 consensus_queue.cc:260] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.412222 29101 ts_tablet_manager.cc:1428] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260929 16:05:11.414400 29099 tablet_bootstrap.cc:492] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987: No bootstrap required, opened a new log
I20260929 16:05:11.414597 29099 ts_tablet_manager.cc:1397] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.007s
I20260929 16:05:11.415220 29099 raft_consensus.cc:348] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.415441 29099 raft_consensus.cc:374] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:11.415544 29099 raft_consensus.cc:729] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf257fb48fb40518f37cead02f53987, State: Initialized, Role: FOLLOWER
I20260929 16:05:11.415809 29099 consensus_queue.cc:260] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.416322 29099 ts_tablet_manager.cc:1428] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260929 16:05:11.590281 29188 raft_consensus.cc:482] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:05:11.590539 29188 raft_consensus.cc:504] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.591331 29188 leader_election.cc:288] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909), 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249)
I20260929 16:05:11.592171 29024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01bc1e9fe6fa405aab2dff32ea58d166" candidate_uuid: "889fd5fa55904271be3b51dcc81a03f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf257fb48fb40518f37cead02f53987" is_pre_election: true
I20260929 16:05:11.592173 28750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01bc1e9fe6fa405aab2dff32ea58d166" candidate_uuid: "889fd5fa55904271be3b51dcc81a03f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" is_pre_election: true
I20260929 16:05:11.592408 29024 raft_consensus.cc:2454] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 889fd5fa55904271be3b51dcc81a03f3 in term 0.
I20260929 16:05:11.592430 28750 raft_consensus.cc:2454] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 889fd5fa55904271be3b51dcc81a03f3 in term 0.
I20260929 16:05:11.592872 28816 leader_election.cc:302] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 889fd5fa55904271be3b51dcc81a03f3, b5b1f07b4e7b4ef48427a2dbbc76987c; no voters:
I20260929 16:05:11.593163 29188 raft_consensus.cc:2781] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:05:11.593271 29188 raft_consensus.cc:482] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:05:11.593384 29188 raft_consensus.cc:3037] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:11.595894 29188 raft_consensus.cc:504] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.596546 29188 leader_election.cc:288] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [CANDIDATE]: Term 1 election: Requested vote from peers b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909), 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249)
I20260929 16:05:11.597153 28750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01bc1e9fe6fa405aab2dff32ea58d166" candidate_uuid: "889fd5fa55904271be3b51dcc81a03f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c"
I20260929 16:05:11.597222 29024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01bc1e9fe6fa405aab2dff32ea58d166" candidate_uuid: "889fd5fa55904271be3b51dcc81a03f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bf257fb48fb40518f37cead02f53987"
I20260929 16:05:11.597393 28750 raft_consensus.cc:3037] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:11.597469 29024 raft_consensus.cc:3037] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:11.599589 28750 raft_consensus.cc:2454] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 889fd5fa55904271be3b51dcc81a03f3 in term 1.
I20260929 16:05:11.600152 28816 leader_election.cc:302] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 889fd5fa55904271be3b51dcc81a03f3, b5b1f07b4e7b4ef48427a2dbbc76987c; no voters:
I20260929 16:05:11.600472 29188 raft_consensus.cc:2781] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:11.600673 29188 raft_consensus.cc:686] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 LEADER]: Becoming Leader. State: Replica: 889fd5fa55904271be3b51dcc81a03f3, State: Running, Role: LEADER
I20260929 16:05:11.600667 29024 raft_consensus.cc:2454] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 889fd5fa55904271be3b51dcc81a03f3 in term 1.
I20260929 16:05:11.600944 29188 consensus_queue.cc:237] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } }
I20260929 16:05:11.604245 28588 catalog_manager.cc:5495] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130). New cstate: current_term: 1 leader_uuid: "889fd5fa55904271be3b51dcc81a03f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "889fd5fa55904271be3b51dcc81a03f3" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 39709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 } health_report { overall_health: UNKNOWN } } }
I20260929 16:05:11.696005 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.688709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:37443 (local address 127.20.121.131:40249)
0929 16:05:11.688885 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:11.688892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:11.689243 (+ 351us) server_negotiation.cc:407] Connection header received
0929 16:05:11.689488 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:11.689495 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:11.689580 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:11.689672 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:11.690357 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.691582 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.693786 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.694487 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.694627 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.695209 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:11.695365 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:11.695448 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:11.695631 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 16:05:11.695713 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 16:05:11.696434 29204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.688294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40249 (local address 127.20.121.130:37443)
0929 16:05:11.689084 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:11.689115 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:11.689338 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:11.689722 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:11.689727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:11.689740 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:05:11.690185 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:11.690201 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.691733 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.691738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:11.693599 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:11.693624 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.694695 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.694707 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:11.694904 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.694995 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:05:11.695479 (+ 484us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:05:11.695495 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 16:05:11.695599 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 16:05:11.695774 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":592,"threads_started":1}
I20260929 16:05:11.706485 29024 raft_consensus.cc:1261] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Refusing update from remote peer 889fd5fa55904271be3b51dcc81a03f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:11.706561 28750 raft_consensus.cc:1261] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Refusing update from remote peer 889fd5fa55904271be3b51dcc81a03f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:11.707269 29188 consensus_queue.cc:1035] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b1f07b4e7b4ef48427a2dbbc76987c" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:11.707635 29194 consensus_queue.cc:1035] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bf257fb48fb40518f37cead02f53987" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:11.739638 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.733653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:42653 (local address 127.20.121.130:39709)
0929 16:05:11.733876 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:11.733883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:11.733946 (+ 63us) server_negotiation.cc:407] Connection header received
0929 16:05:11.734240 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:11.734246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:11.734335 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:11.734422 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:11.734922 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.735719 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.737748 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.738330 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.738444 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.738945 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:11.739048 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:11.739119 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:11.739288 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 16:05:11.739351 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260929 16:05:11.740034 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.733275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:39709 (local address 127.20.121.131:42653)
0929 16:05:11.733857 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:11.733877 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:11.734090 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:11.734475 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:11.734480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:11.734493 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 16:05:11.734759 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:11.734773 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.735887 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.735892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:11.737482 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:11.737499 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.738511 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.738517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:11.738736 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.738789 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 16:05:11.739163 (+ 374us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 16:05:11.739175 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 16:05:11.739264 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 16:05:11.739420 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":421,"threads_started":1}
I20260929 16:05:11.773713 28588 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:11.774003 28588 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:11.777241 28588 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d45d86ba489343f3b1b5cfe5a264d68c: Sending DeleteTablet for 3 replicas of tablet 01bc1e9fe6fa405aab2dff32ea58d166
I20260929 16:05:11.778226 28725 tablet_service.cc:1514] Processing DeleteTablet for tablet 01bc1e9fe6fa405aab2dff32ea58d166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:11 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:46954
I20260929 16:05:11.778223 28865 tablet_service.cc:1514] Processing DeleteTablet for tablet 01bc1e9fe6fa405aab2dff32ea58d166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:11 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:47546
I20260929 16:05:11.778509 29169 tablet_replica.cc:330] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3: stopping tablet replica
I20260929 16:05:11.778499 29168 tablet_replica.cc:330] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c: stopping tablet replica
I20260929 16:05:11.778680 29168 raft_consensus.cc:2229] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:11.778687 29169 raft_consensus.cc:2229] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:05:11.778885 29168 raft_consensus.cc:2258] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:11.779037 29169 raft_consensus.cc:2258] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:11.779659 29004 tablet_service.cc:1514] Processing DeleteTablet for tablet 01bc1e9fe6fa405aab2dff32ea58d166 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:11 UTC) from {username='oryx', principal='oryx/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:44762
I20260929 16:05:11.780221 29169 ts_tablet_manager.cc:1905] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:11.780884 29168 ts_tablet_manager.cc:1905] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:11.781404 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 28659
I20260929 16:05:11.782186 29170 tablet_replica.cc:330] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987: stopping tablet replica
I20260929 16:05:11.782382 29170 raft_consensus.cc:2229] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:11.782590 29170 raft_consensus.cc:2258] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:11.785149 29170 ts_tablet_manager.cc:1905] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:11.787658 29168 ts_tablet_manager.cc:1918] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:05:11.787784 29168 log.cc:1196] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/01bc1e9fe6fa405aab2dff32ea58d166
I20260929 16:05:11.788193 29168 ts_tablet_manager.cc:1939] T 01bc1e9fe6fa405aab2dff32ea58d166 P b5b1f07b4e7b4ef48427a2dbbc76987c: Deleting consensus metadata
I20260929 16:05:11.789528 28573 catalog_manager.cc:4841] TS b5b1f07b4e7b4ef48427a2dbbc76987c (127.20.121.129:40909): tablet 01bc1e9fe6fa405aab2dff32ea58d166 (table test-table [id=d01b10df848948afbb2d082c23cfecb7]) successfully deleted
I20260929 16:05:11.791719 29170 ts_tablet_manager.cc:1918] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:05:11.791872 29170 log.cc:1196] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/01bc1e9fe6fa405aab2dff32ea58d166
I20260929 16:05:11.792341 29170 ts_tablet_manager.cc:1939] T 01bc1e9fe6fa405aab2dff32ea58d166 P 8bf257fb48fb40518f37cead02f53987: Deleting consensus metadata
I20260929 16:05:11.793393 28573 catalog_manager.cc:4841] TS 8bf257fb48fb40518f37cead02f53987 (127.20.121.131:40249): tablet 01bc1e9fe6fa405aab2dff32ea58d166 (table test-table [id=d01b10df848948afbb2d082c23cfecb7]) successfully deleted
I20260929 16:05:11.799419 29169 ts_tablet_manager.cc:1918] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 16:05:11.799615 29169 log.cc:1196] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/01bc1e9fe6fa405aab2dff32ea58d166
I20260929 16:05:11.800083 29169 ts_tablet_manager.cc:1939] T 01bc1e9fe6fa405aab2dff32ea58d166 P 889fd5fa55904271be3b51dcc81a03f3: Deleting consensus metadata
I20260929 16:05:11.801180 28575 catalog_manager.cc:4841] TS 889fd5fa55904271be3b51dcc81a03f3 (127.20.121.130:39709): tablet 01bc1e9fe6fa405aab2dff32ea58d166 (table test-table [id=d01b10df848948afbb2d082c23cfecb7]) successfully deleted
Sep 29 16:05:11 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.129@KRBTEST.COM
I20260929 16:05:11.814697 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:11.783967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46976 (local address 127.20.121.129:40909)
0929 16:05:11.789444 (+ 5477us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:11.789451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:11.798607 (+ 9156us) server_negotiation.cc:407] Connection header received
0929 16:05:11.798825 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:11.798831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:11.798916 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:11.799022 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:11.800694 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.801636 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.803459 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:11.803882 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:11.803977 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:11.808613 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:11.808628 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:11.808644 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:11.808687 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:11.811871 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:11.812575 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:11.812582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:11.812584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:11.812714 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:11.813092 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:11.813101 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:11.813103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:11.813434 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:11.813753 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:11.814104 (+ 351us) server_negotiation.cc:299] Negotiation successful
0929 16:05:11.814328 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5334,"thread_start_us":5223,"threads_started":1}
I20260929 16:05:11.815325 28790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:11.978268 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 28659
I20260929 16:05:12.002323 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 28799
Sep 29 16:05:12 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.130@KRBTEST.COM
I20260929 16:05:12.017105 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:12.003017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47556 (local address 127.20.121.130:39709)
0929 16:05:12.003214 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:12.003222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:12.003243 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:05:12.003456 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:12.003460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:12.003524 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:12.003642 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:12.004799 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:12.005482 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:12.006831 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:12.007242 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:12.007370 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:12.011661 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:12.011677 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:12.011695 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:12.011740 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:12.014768 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:12.015433 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:12.015440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:12.015443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:12.015570 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:12.015855 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:12.015858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:12.015860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:12.016195 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:12.016413 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:12.016676 (+ 263us) server_negotiation.cc:299] Negotiation successful
0929 16:05:12.016838 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 16:05:12.017831 28930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:12.144984 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 28799
I20260929 16:05:12.170305 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 28939
Sep 29 16:05:12 dist-test-slave-x07d krb5kdc[28543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.121.131@KRBTEST.COM
I20260929 16:05:12.186165 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:12.171187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44796 (local address 127.20.121.131:40249)
0929 16:05:12.171424 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:12.171431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:12.171453 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:05:12.171512 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:12.171517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:12.171582 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:12.171666 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:12.172923 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:12.173628 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:12.175008 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:12.175429 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:12.175560 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:12.180209 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:12.180225 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:12.180243 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:12.180283 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:12.183285 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:12.184180 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:12.184193 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:12.184200 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:12.184362 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:12.184757 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:12.184760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:12.184762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:12.185130 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:12.185347 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:12.185670 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 16:05:12.185840 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260929 16:05:12.186821 29070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:12.330960 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 28939
I20260929 16:05:12.358752 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 28556
I20260929 16:05:12.360122 28628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:12.478972 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 28556
2026-09-29T16:05:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6486 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:05:12 dist-test-slave-x07d krb5kdc[29219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:05:12 dist-test-slave-x07d krb5kdc[29219](info): set up 2 sockets
Sep 29 16:05:12 dist-test-slave-x07d krb5kdc[29219](info): commencing operation
krb5kdc: starting...
W20260929 16:05:13.691110 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:05:13 dist-test-slave-x07d krb5kdc[29219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:05:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:05:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:05:13.941895 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:36743
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:36743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:14.280874 29232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:14.281267 29232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:14.281440 29232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:14.288837 29232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:05:14.289019 29232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:14.289134 29232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:14.289240 29232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:05:14.289336 29232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:05:14.299347 29232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:36743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:36743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:14.300779 29232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:14.302206 29232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:14.313939 29237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:14.314026 29240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:14.314126 29238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:14.315636 29232 server_base.cc:1034] running on GCE node
I20260929 16:05:14.316504 29232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:14.317974 29232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:14.319267 29232 hybrid_clock.cc:648] HybridClock initialized: now 1790697914319211 us; error 37 us; skew 500 ppm
I20260929 16:05:14.319718 29232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T16:05:14Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1908 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:05:14 dist-test-slave-x07d krb5kdc[29246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:05:14 dist-test-slave-x07d krb5kdc[29246](info): set up 2 sockets
Sep 29 16:05:14 dist-test-slave-x07d krb5kdc[29246](info): commencing operation
krb5kdc: starting...
W20260929 16:05:15.511103 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:05:15 dist-test-slave-x07d krb5kdc[29246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:05:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:05:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:05:15.754449 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:45651
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:45651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:16.100152 29259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:16.100486 29259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:16.100592 29259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:16.108817 29259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:05:16.108960 29259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:16.109017 29259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:16.109081 29259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:05:16.109133 29259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:05:16.120409 29259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:45651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:16.121816 29259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:16.123415 29259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:16.136549 29264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:16.136581 29265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:16.136619 29267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:16.137182 29259 server_base.cc:1034] running on GCE node
I20260929 16:05:16.138113 29259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:16.139619 29259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:16.140870 29259 hybrid_clock.cc:648] HybridClock initialized: now 1790697916140825 us; error 68 us; skew 500 ppm
I20260929 16:05:16.141234 29259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:16 dist-test-slave-x07d krb5kdc[29246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697916, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:16.154281 29259 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:05:16.156005 29259 webserver.cc:460] Webserver started at http://127.20.121.190:39667/ using document root <none> and password file <none>
I20260929 16:05:16.156840 29259 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:16.157001 29259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:16.157323 29259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:16.160491 29259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "06c2aa33671d44529405c36e8593165d"
format_stamp: "Formatted at 2026-09-29 16:05:16 on dist-test-slave-x07d"
server_key: "9a5ac017bd186cc9a0b8a67255da0185"
server_key_iv: "e2ede74bc1b9734da2cdabcfded6b782"
server_key_version: "encryptionkey@0"
I20260929 16:05:16.161289 29259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "06c2aa33671d44529405c36e8593165d"
format_stamp: "Formatted at 2026-09-29 16:05:16 on dist-test-slave-x07d"
server_key: "9a5ac017bd186cc9a0b8a67255da0185"
server_key_iv: "e2ede74bc1b9734da2cdabcfded6b782"
server_key_version: "encryptionkey@0"
I20260929 16:05:16.167322 29259 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260929 16:05:16.171188 29274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:16.172824 29259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 16:05:16.173050 29259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "06c2aa33671d44529405c36e8593165d"
format_stamp: "Formatted at 2026-09-29 16:05:16 on dist-test-slave-x07d"
server_key: "9a5ac017bd186cc9a0b8a67255da0185"
server_key_iv: "e2ede74bc1b9734da2cdabcfded6b782"
server_key_version: "encryptionkey@0"
I20260929 16:05:16.173266 29259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:16.196631 29259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:16.201470 29259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:16.201823 29259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:16.223119 29259 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:45651
I20260929 16:05:16.223121 29325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:45651 every 8 connection(s)
I20260929 16:05:16.224699 29259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:05:16.227937 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 29259
I20260929 16:05:16.228195 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:05:16.228863 29326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:16.228961 20966 external_mini_cluster.cc:1428] Setting key b070ea3d973246e38a928c587ff02baf
I20260929 16:05:16.244542 29326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d: Bootstrap starting.
I20260929 16:05:16.247897 29326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:16.249128 29326 log.cc:824] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d: Log is configured to *not* fsync() on all Append() calls
Sep 29 16:05:16 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:05:16.252763 29326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d: No bootstrap required, opened a new log
I20260929 16:05:16.263156 29329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:16.231255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36818 (local address 127.20.121.190:45651)
0929 16:05:16.232038 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:16.232052 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:16.232099 (+ 47us) server_negotiation.cc:407] Connection header received
0929 16:05:16.233713 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:16.233737 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:16.234309 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:16.234717 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:16.236688 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:16.238267 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:16.240092 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:16.240822 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:16.240979 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:16.253431 (+ 12452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:16.253443 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:16.253464 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:16.253528 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:16.257865 (+ 4337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:16.258783 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:16.258793 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:16.258803 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:16.259040 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:16.259434 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:16.259441 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:16.259450 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:16.260289 (+ 839us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:16.260600 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:16.261020 (+ 420us) server_negotiation.cc:299] Negotiation successful
0929 16:05:16.261587 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":323,"threads_started":1}
I20260929 16:05:16.264081 29326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } }
I20260929 16:05:16.264557 29326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:16.264802 29326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06c2aa33671d44529405c36e8593165d, State: Initialized, Role: FOLLOWER
I20260929 16:05:16.265486 29326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } }
I20260929 16:05:16.265760 29326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:05:16.265913 29326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:05:16.266148 29326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:16.268764 29326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } }
I20260929 16:05:16.269359 29326 leader_election.cc:302] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06c2aa33671d44529405c36e8593165d; no voters:
I20260929 16:05:16.270084 29326 leader_election.cc:288] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:05:16.270252 29331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:16.270937 29331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 1 LEADER]: Becoming Leader. State: Replica: 06c2aa33671d44529405c36e8593165d, State: Running, Role: LEADER
I20260929 16:05:16.271481 29331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } }
I20260929 16:05:16.272276 29326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:05:16.274407 29333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06c2aa33671d44529405c36e8593165d. Latest consensus state: current_term: 1 leader_uuid: "06c2aa33671d44529405c36e8593165d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } } }
I20260929 16:05:16.274814 29333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:16.274873 29332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06c2aa33671d44529405c36e8593165d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } } }
I20260929 16:05:16.275251 29332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:16.281704 29339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:05:16.286291 29339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:05:16.294409 29339 catalog_manager.cc:1347] Generated new cluster ID: 9d79a503553441d7aeb57a96b4251b40
I20260929 16:05:16.294538 29339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:05:16.309564 29339 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:05:16.311568 29339 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:05:16.330925 29339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d: Generated new TSK 0
I20260929 16:05:16.331961 29339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:05:16.410585 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45651
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:16.759035 29352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:16.759342 29352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:16.759513 29352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:16.768208 29352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:16.768486 29352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:16.768697 29352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:05:16.780798 29352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:16.782366 29352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:16.783815 29352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:16.799104 29360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:16.799177 29357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:16.799139 29358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:16.800199 29352 server_base.cc:1034] running on GCE node
I20260929 16:05:16.800887 29352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:16.802150 29352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:16.803426 29352 hybrid_clock.cc:648] HybridClock initialized: now 1790697916803360 us; error 58 us; skew 500 ppm
I20260929 16:05:16.803821 29352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:16 dist-test-slave-x07d krb5kdc[29246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697916, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:16.807530 29352 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:05:16.809321 29352 webserver.cc:460] Webserver started at http://127.20.121.129:45717/ using document root <none> and password file <none>
I20260929 16:05:16.810117 29352 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:16.810262 29352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:16.810570 29352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:16.813458 29352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "1271760b267c41d4bead5b18979e2d88"
format_stamp: "Formatted at 2026-09-29 16:05:16 on dist-test-slave-x07d"
server_key: "c7568cef5c9a2b04317e449987f4e483"
server_key_iv: "a58fd6b9d19e212314e3212a6504b6f9"
server_key_version: "encryptionkey@0"
I20260929 16:05:16.814363 29352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "1271760b267c41d4bead5b18979e2d88"
format_stamp: "Formatted at 2026-09-29 16:05:16 on dist-test-slave-x07d"
server_key: "c7568cef5c9a2b04317e449987f4e483"
server_key_iv: "a58fd6b9d19e212314e3212a6504b6f9"
server_key_version: "encryptionkey@0"
I20260929 16:05:16.819507 29352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 16:05:16.822966 29367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:16.824250 29352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 16:05:16.824431 29352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "1271760b267c41d4bead5b18979e2d88"
format_stamp: "Formatted at 2026-09-29 16:05:16 on dist-test-slave-x07d"
server_key: "c7568cef5c9a2b04317e449987f4e483"
server_key_iv: "a58fd6b9d19e212314e3212a6504b6f9"
server_key_version: "encryptionkey@0"
I20260929 16:05:16.824633 29352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:16.849833 29352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:16.853880 29352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:16.854226 29352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:16.855226 29352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:16.856588 29352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:16.856691 29352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:16.856840 29352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:16.856912 29352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:16.890328 29352 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:46833
I20260929 16:05:16.890410 29479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:46833 every 8 connection(s)
I20260929 16:05:16.891613 29352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:05:16.897352 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 29352
I20260929 16:05:16.897603 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:05:16.898365 20966 external_mini_cluster.cc:1428] Setting key ed7ca6c576b0012e1b546eb3addecea9
Sep 29 16:05:16 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697916, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:05:16.913262 29482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:16.894427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:42217 (local address 127.20.121.190:45651)
0929 16:05:16.894931 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:16.894937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:16.895831 (+ 894us) server_negotiation.cc:407] Connection header received
0929 16:05:16.897686 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:16.897692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:16.897792 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:16.897936 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:16.899782 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:16.900821 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:16.902455 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:16.903183 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:16.903334 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:16.907116 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:16.907120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:16.907124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:16.907155 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:16.909678 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:16.910694 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:16.910698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:16.910700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:16.910824 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:16.911331 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:16.911335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:16.911337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:16.911586 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:16.911691 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:16.912740 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0929 16:05:16.912990 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":232,"threads_started":1}
I20260929 16:05:16.914841 29483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:16.894975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:42217)
0929 16:05:16.895672 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:16.895708 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:16.897361 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:16.898207 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:16.898215 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:16.898796 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:16.899611 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:16.899636 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:16.900980 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:16.900983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:16.902314 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:16.902326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:16.903438 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:16.903443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:16.903663 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:16.904487 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:16.904511 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:16.906860 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:16.909837 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:16.909849 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:16.909872 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:16.910564 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:16.910943 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:16.910950 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:16.910956 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:16.911218 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:16.911716 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:16.911728 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:16.912223 (+ 495us) client_negotiation.cc:769] Sending connection context
0929 16:05:16.912550 (+ 327us) client_negotiation.cc:240] Negotiation successful
0929 16:05:16.913162 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":272,"threads_started":1}
I20260929 16:05:16.916615 29480 heartbeater.cc:347] Connected to a master server at 127.20.121.190:45651
I20260929 16:05:16.916939 29480 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:16.917693 29480 heartbeater.cc:511] Master 127.20.121.190:45651 requested a full tablet report, sending...
I20260929 16:05:16.920277 29291 ts_manager.cc:194] Registered new tserver with Master: 1271760b267c41d4bead5b18979e2d88 (127.20.121.129:46833)
I20260929 16:05:16.922531 29291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:42217
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:05:16.952865 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45651
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:17.296981 29486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:17.297286 29486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:17.297381 29486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:17.304692 29486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:17.304814 29486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:17.304936 29486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:05:17.314976 29486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:17.316283 29486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:17.317531 29486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:17.330715 29492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:17.330914 29491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:17.331449 29494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:17.332067 29486 server_base.cc:1034] running on GCE node
I20260929 16:05:17.332772 29486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:17.333833 29486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:17.335055 29486 hybrid_clock.cc:648] HybridClock initialized: now 1790697917334978 us; error 66 us; skew 500 ppm
I20260929 16:05:17.335444 29486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:17 dist-test-slave-x07d krb5kdc[29246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:17.339155 29486 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:05:17.340867 29486 webserver.cc:460] Webserver started at http://127.20.121.130:37903/ using document root <none> and password file <none>
I20260929 16:05:17.341629 29486 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:17.341768 29486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:17.342214 29486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:17.345687 29486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "b62509bc6fb348099f9002bd04339a74"
format_stamp: "Formatted at 2026-09-29 16:05:17 on dist-test-slave-x07d"
server_key: "ca6d430cf608ce978c9785461381f824"
server_key_iv: "4e51724bf70f7ba592144b29eee864bb"
server_key_version: "encryptionkey@0"
I20260929 16:05:17.346668 29486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "b62509bc6fb348099f9002bd04339a74"
format_stamp: "Formatted at 2026-09-29 16:05:17 on dist-test-slave-x07d"
server_key: "ca6d430cf608ce978c9785461381f824"
server_key_iv: "4e51724bf70f7ba592144b29eee864bb"
server_key_version: "encryptionkey@0"
I20260929 16:05:17.352586 29486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:05:17.356734 29501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:17.358397 29486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 16:05:17.358595 29486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "b62509bc6fb348099f9002bd04339a74"
format_stamp: "Formatted at 2026-09-29 16:05:17 on dist-test-slave-x07d"
server_key: "ca6d430cf608ce978c9785461381f824"
server_key_iv: "4e51724bf70f7ba592144b29eee864bb"
server_key_version: "encryptionkey@0"
I20260929 16:05:17.358806 29486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:17.371536 29486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:17.376109 29486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:17.376423 29486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:17.377516 29486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:17.378878 29486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:17.378949 29486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:17.379096 29486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:17.379163 29486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:17.413954 29486 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:41687
I20260929 16:05:17.414136 29613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:41687 every 8 connection(s)
I20260929 16:05:17.415408 29486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:05:17.416182 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 29486
I20260929 16:05:17.416379 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:05:17.416952 20966 external_mini_cluster.cc:1428] Setting key e0476926dc22e4bda6bdaf6c39abd20e
Sep 29 16:05:17 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:05:17.439028 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:17.418404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:50435 (local address 127.20.121.190:45651)
0929 16:05:17.418986 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:17.418990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:17.420240 (+ 1250us) server_negotiation.cc:407] Connection header received
0929 16:05:17.422202 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:17.422211 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:17.422310 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:17.422427 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:17.424087 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:17.425264 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:17.427099 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:17.427560 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:17.427669 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:17.432323 (+ 4654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:17.432329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:17.432332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:17.432368 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:17.435093 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:17.436039 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:17.436047 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:17.436052 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:17.436235 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:17.436716 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:17.436724 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:17.436731 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:17.437147 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:17.437319 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:17.438076 (+ 757us) server_negotiation.cc:299] Negotiation successful
0929 16:05:17.438323 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":235,"threads_started":1}
I20260929 16:05:17.439647 29617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:17.419171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:50435)
0929 16:05:17.420042 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:17.420089 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:17.421837 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:17.422696 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:17.422706 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:17.423260 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:17.423883 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:17.423902 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:17.425454 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:17.425458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:17.426907 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:17.426926 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:17.427819 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:17.427828 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:17.428175 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:17.429009 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:17.429046 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:17.432033 (+ 2987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:17.435277 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:17.435287 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:17.435308 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:17.435839 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:17.436369 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:17.436372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:17.436375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:17.436570 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:17.437286 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:17.437292 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:17.437644 (+ 352us) client_negotiation.cc:769] Sending connection context
0929 16:05:17.437886 (+ 242us) client_negotiation.cc:240] Negotiation successful
0929 16:05:17.438445 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":374,"threads_started":1}
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
I20260929 16:05:17.441270 29614 heartbeater.cc:347] Connected to a master server at 127.20.121.190:45651
I20260929 16:05:17.441635 29614 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:17.442407 29614 heartbeater.cc:511] Master 127.20.121.190:45651 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
I20260929 16:05:17.444087 29291 ts_manager.cc:194] Registered new tserver with Master: b62509bc6fb348099f9002bd04339a74 (127.20.121.130:41687)
I20260929 16:05:17.444918 29291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:50435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:05:17.474882 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45651
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:17.817883 29620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:17.818264 29620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:17.818429 29620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:17.825834 29620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:17.825954 29620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:17.826201 29620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:05:17.836262 29620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:17.837611 29620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:17.839003 29620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:17.852566 29625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:17.852634 29628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:17.852665 29626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:17.853828 29620 server_base.cc:1034] running on GCE node
I20260929 16:05:17.854540 29620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:17.855553 29620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:17.856786 29620 hybrid_clock.cc:648] HybridClock initialized: now 1790697917856728 us; error 70 us; skew 500 ppm
I20260929 16:05:17.857125 29620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:17 dist-test-slave-x07d krb5kdc[29246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:17.860260 29620 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:05:17.861789 29620 webserver.cc:460] Webserver started at http://127.20.121.131:45573/ using document root <none> and password file <none>
I20260929 16:05:17.862619 29620 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:17.862761 29620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:17.863133 29620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:17.866164 29620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "3c9496aa17ec45c4af5b8a5232fc34ab"
format_stamp: "Formatted at 2026-09-29 16:05:17 on dist-test-slave-x07d"
server_key: "4ba741bacc183a5b88c3880d69125a66"
server_key_iv: "35cf873695481eb788e79909d5dd8e05"
server_key_version: "encryptionkey@0"
I20260929 16:05:17.866969 29620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "3c9496aa17ec45c4af5b8a5232fc34ab"
format_stamp: "Formatted at 2026-09-29 16:05:17 on dist-test-slave-x07d"
server_key: "4ba741bacc183a5b88c3880d69125a66"
server_key_iv: "35cf873695481eb788e79909d5dd8e05"
server_key_version: "encryptionkey@0"
I20260929 16:05:17.872412 29620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:05:17.876684 29635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:17.878611 29620 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260929 16:05:17.878805 29620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "3c9496aa17ec45c4af5b8a5232fc34ab"
format_stamp: "Formatted at 2026-09-29 16:05:17 on dist-test-slave-x07d"
server_key: "4ba741bacc183a5b88c3880d69125a66"
server_key_iv: "35cf873695481eb788e79909d5dd8e05"
server_key_version: "encryptionkey@0"
I20260929 16:05:17.879042 29620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:17.899665 29620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:17.904235 29620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:17.904546 29620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:17.905534 29620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:17.906992 29620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:17.907063 29620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:17.907233 29620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:17.907284 29620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:17.925730 29480 heartbeater.cc:503] Master 127.20.121.190:45651 was elected leader, sending a full tablet report...
I20260929 16:05:17.941934 29620 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:43421
I20260929 16:05:17.941982 29747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:43421 every 8 connection(s)
I20260929 16:05:17.943279 29620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:05:17.949541 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 29620
I20260929 16:05:17.949738 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:05:17.950241 20966 external_mini_cluster.cc:1428] Setting key 618d6b90e6321071a2e9a2274338704c
Sep 29 16:05:17 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:05:17.963311 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:17.945824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:34163 (local address 127.20.121.190:45651)
0929 16:05:17.946328 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:17.946335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:17.947279 (+ 944us) server_negotiation.cc:407] Connection header received
0929 16:05:17.949063 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:17.949067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:17.949139 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:17.949233 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:17.951453 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:17.952229 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:17.953735 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:17.954191 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:17.954322 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:17.957631 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:17.957636 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:17.957640 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:17.957671 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:17.960210 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:17.960944 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:17.960949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:17.960953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:17.961075 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:17.961515 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:17.961521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:17.961525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:17.961792 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:17.961943 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:17.962711 (+ 768us) server_negotiation.cc:299] Negotiation successful
0929 16:05:17.962920 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":265,"threads_started":1}
I20260929 16:05:17.964490 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:17.946426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:34163)
0929 16:05:17.947111 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:17.947157 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:17.948758 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:17.949951 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:17.949959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:17.950607 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:17.951288 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:17.951305 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:17.952390 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:17.952395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:17.953597 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:17.953608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:17.954411 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:17.954418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:17.954655 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:17.955261 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:17.955287 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:17.957405 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:17.960352 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:17.960361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:17.960378 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:17.960817 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:17.961186 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:17.961191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:17.961194 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:17.961386 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:17.961920 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:17.961928 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:17.962348 (+ 420us) client_negotiation.cc:769] Sending connection context
0929 16:05:17.962639 (+ 291us) client_negotiation.cc:240] Negotiation successful
0929 16:05:17.963222 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":264,"threads_started":1}
I20260929 16:05:17.966177 29748 heartbeater.cc:347] Connected to a master server at 127.20.121.190:45651
I20260929 16:05:17.966579 29748 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:17.967432 29748 heartbeater.cc:511] Master 127.20.121.190:45651 requested a full tablet report, sending...
I20260929 16:05:17.969146 29291 ts_manager.cc:194] Registered new tserver with Master: 3c9496aa17ec45c4af5b8a5232fc34ab (127.20.121.131:43421)
I20260929 16:05:17.969940 29291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:34163
I20260929 16:05:17.978371 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab.
Entry for principal oryx/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab.
WARNING: no policy specified for oryx/127.20.121.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab.
Entry for principal oryx/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab.
WARNING: no policy specified for oryx/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab.
Entry for principal oryx/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab.
I20260929 16:05:18.125978 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 29352
Sep 29 16:05:18 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:05:18.142870 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:18.127296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57194 (local address 127.20.121.129:46833)
0929 16:05:18.127965 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:18.127972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:18.128507 (+ 535us) server_negotiation.cc:407] Connection header received
0929 16:05:18.128681 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:18.128688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:18.128903 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:18.129140 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:18.130462 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:18.131494 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:18.132670 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:18.133217 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:18.133311 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:18.137423 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:18.137435 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:18.137450 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:18.137488 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:18.140391 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:18.141066 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:18.141073 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:18.141075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:18.141204 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:18.141515 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:18.141518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:18.141520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:18.141849 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:18.142156 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:18.142393 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 16:05:18.142588 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":380,"threads_started":1}
I20260929 16:05:18.144191 29475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:18.238462 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 29352
I20260929 16:05:18.256217 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 29486
Sep 29 16:05:18 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:05:18.273181 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:18.257208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.20.121.130:41687)
0929 16:05:18.257774 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:18.257781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:18.257806 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:05:18.257863 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:18.257870 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:18.258173 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:18.258298 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:18.259460 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:18.260491 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:18.261827 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:18.262491 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:18.262599 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:18.266997 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:18.267011 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:18.267027 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:18.267070 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:18.270258 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:18.271064 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:18.271071 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:18.271074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:18.271208 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:18.271598 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:18.271602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:18.271604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:18.271953 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:18.272180 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:18.272542 (+ 362us) server_negotiation.cc:299] Negotiation successful
0929 16:05:18.272798 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":269,"threads_started":1}
I20260929 16:05:18.274701 29609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:18.357388 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 29486
I20260929 16:05:18.373397 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 29620
Sep 29 16:05:18 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:05:18.391944 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:18.374556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.20.121.131:43421)
0929 16:05:18.375172 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:18.375179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:18.375208 (+ 29us) server_negotiation.cc:407] Connection header received
0929 16:05:18.375266 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:18.375272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:18.375517 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:18.375708 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:18.377134 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:18.378337 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:18.379716 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:18.380281 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:18.380444 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:18.385575 (+ 5131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:18.385589 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:18.385605 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:18.385644 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:18.388960 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:18.389680 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:18.389687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:18.389689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:18.389893 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:18.390345 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:18.390348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:18.390351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:18.390712 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:18.391003 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:18.391262 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 16:05:18.391529 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":306,"threads_started":1}
I20260929 16:05:18.393306 29743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:18.478421 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 29620
I20260929 16:05:18.494663 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 29259
I20260929 16:05:18.496044 29321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:18.599251 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 29259
I20260929 16:05:18.620654 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:45651
--webserver_interface=127.20.121.190
--webserver_port=39667
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:45651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:18.962829 29766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:18.963131 29766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:18.963220 29766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:18.970459 29766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:05:18.970582 29766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:18.970626 29766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:18.970683 29766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:05:18.970726 29766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:05:18.981307 29766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:45651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=39667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:18.982774 29766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:18.984040 29766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:18.996872 29772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:18.996985 29771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:18.996939 29774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:18.997984 29766 server_base.cc:1034] running on GCE node
I20260929 16:05:18.998983 29766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:19.000584 29766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:19.001832 29766 hybrid_clock.cc:648] HybridClock initialized: now 1790697919001766 us; error 59 us; skew 500 ppm
I20260929 16:05:19.002295 29766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:19 dist-test-slave-x07d krb5kdc[29246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697919, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:19.006188 29766 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:05:19.008525 29766 webserver.cc:460] Webserver started at http://127.20.121.190:39667/ using document root <none> and password file <none>
I20260929 16:05:19.009328 29766 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:19.009500 29766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:19.015614 29766 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 16:05:19.018801 29781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:19.020539 29766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:05:19.020748 29766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "06c2aa33671d44529405c36e8593165d"
format_stamp: "Formatted at 2026-09-29 16:05:16 on dist-test-slave-x07d"
server_key: "9a5ac017bd186cc9a0b8a67255da0185"
server_key_iv: "e2ede74bc1b9734da2cdabcfded6b782"
server_key_version: "encryptionkey@0"
I20260929 16:05:19.021791 29766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:19.037658 29766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:19.041751 29766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:19.042169 29766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:19.060855 29766 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:45651
I20260929 16:05:19.060878 29832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:45651 every 8 connection(s)
I20260929 16:05:19.062425 29766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:05:19.063846 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 29766
I20260929 16:05:19.064823 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:46833
--local_ip_for_outbound_sockets=127.20.121.129
--tserver_master_addrs=127.20.121.190:45651
--webserver_port=45717
--webserver_interface=127.20.121.129
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.121.129
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:05:19.073199 29833 sys_catalog.cc:263] Verifying existing consensus state
I20260929 16:05:19.076972 29833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d: Bootstrap starting.
I20260929 16:05:19.097589 29833 log.cc:824] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:19.107653 29833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 16:05:19.108300 29833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d: Bootstrap complete.
I20260929 16:05:19.118407 29833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } }
I20260929 16:05:19.119318 29833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06c2aa33671d44529405c36e8593165d, State: Initialized, Role: FOLLOWER
I20260929 16:05:19.120095 29833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } }
I20260929 16:05:19.120371 29833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:05:19.120522 29833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:05:19.120729 29833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 1 FOLLOWER]: Advancing to term 2
I20260929 16:05:19.125134 29833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } }
I20260929 16:05:19.125780 29833 leader_election.cc:302] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06c2aa33671d44529405c36e8593165d; no voters:
I20260929 16:05:19.126600 29833 leader_election.cc:288] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 16:05:19.126748 29837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 2 FOLLOWER]: Leader election won for term 2
I20260929 16:05:19.129386 29833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:05:19.130354 29837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [term 2 LEADER]: Becoming Leader. State: Replica: 06c2aa33671d44529405c36e8593165d, State: Running, Role: LEADER
I20260929 16:05:19.131072 29837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } }
I20260929 16:05:19.134311 29839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06c2aa33671d44529405c36e8593165d. Latest consensus state: current_term: 2 leader_uuid: "06c2aa33671d44529405c36e8593165d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } } }
I20260929 16:05:19.134852 29839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:19.135571 29838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "06c2aa33671d44529405c36e8593165d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c2aa33671d44529405c36e8593165d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45651 } } }
I20260929 16:05:19.135970 29838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:19.143147 29846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:05:19.151553 29846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:05:19.153698 29846 catalog_manager.cc:1259] Loaded cluster ID: 9d79a503553441d7aeb57a96b4251b40
I20260929 16:05:19.153861 29846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:05:19.159969 29846 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:05:19.162662 29846 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 06c2aa33671d44529405c36e8593165d: Loaded TSK: 0
I20260929 16:05:19.165417 29846 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 16:05:19.449117 29835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:19.449625 29835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:19.449834 29835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:19.459089 29835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:19.459295 29835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:19.459582 29835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:05:19.471190 29835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.129.keytab
--principal=oryx/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:46833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=45717
--tserver_master_addrs=127.20.121.190:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:19.472672 29835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:19.474252 29835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:19.490615 29860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:19.490638 29861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:19.491328 29863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:19.491899 29835 server_base.cc:1034] running on GCE node
I20260929 16:05:19.492632 29835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:19.493762 29835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:19.494993 29835 hybrid_clock.cc:648] HybridClock initialized: now 1790697919494938 us; error 54 us; skew 500 ppm
I20260929 16:05:19.495371 29835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:19 dist-test-slave-x07d krb5kdc[29246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697919, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:19.499121 29835 init.cc:377] Logged in from keytab as oryx/127.20.121.129@KRBTEST.COM (short username oryx)
I20260929 16:05:19.501073 29835 webserver.cc:460] Webserver started at http://127.20.121.129:45717/ using document root <none> and password file <none>
I20260929 16:05:19.501874 29835 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:19.502090 29835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:19.508309 29835 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260929 16:05:19.511657 29870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:19.513331 29835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:05:19.513540 29835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "1271760b267c41d4bead5b18979e2d88"
format_stamp: "Formatted at 2026-09-29 16:05:16 on dist-test-slave-x07d"
server_key: "c7568cef5c9a2b04317e449987f4e483"
server_key_iv: "a58fd6b9d19e212314e3212a6504b6f9"
server_key_version: "encryptionkey@0"
I20260929 16:05:19.514672 29835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:19.532249 29835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:19.537130 29835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:19.537428 29835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:19.538496 29835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:19.539894 29835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:19.539987 29835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:19.540148 29835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:19.540253 29835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:19.573872 29835 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:46833
I20260929 16:05:19.573930 29982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:46833 every 8 connection(s)
I20260929 16:05:19.575446 29835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:05:19.582513 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 29835
I20260929 16:05:19.583252 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:41687
--local_ip_for_outbound_sockets=127.20.121.130
--tserver_master_addrs=127.20.121.190:45651
--webserver_port=37903
--webserver_interface=127.20.121.130
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.121.130
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:05:19 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:19 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:19.596737 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:19.579034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:48013)
0929 16:05:19.579889 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:19.579943 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:19.581692 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:19.583649 (+ 1957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:19.583661 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:19.584254 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:19.584800 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:19.584819 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.587251 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.587256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:19.589232 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:19.589248 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.590774 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.590782 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:19.591095 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:19.591941 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:19.591966 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:19.595713 (+ 3747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":382,"threads_started":1}
W20260929 16:05:19.597461 29983 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:45651 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.121.190:45651: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
W20260929 16:05:19.597519 29985 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:19.578968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:48013 (local address 127.20.121.190:45651)
0929 16:05:19.579707 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:19.579719 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:19.580084 (+ 365us) server_negotiation.cc:407] Connection header received
0929 16:05:19.582457 (+ 2373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:19.582479 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:19.582962 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:19.583400 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:19.585031 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.587017 (+ 1986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.589420 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.590494 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.590647 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:19.596240 (+ 5593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:48013: BlockingRecv error: recv got EOF from 127.20.121.129:48013 (error 108)
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":315,"threads_started":1}
Sep 29 16:05:19 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:19 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:19.608435 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:19.598294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:52615)
0929 16:05:19.598516 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:19.598538 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:19.598658 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:19.599067 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:19.599074 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:19.599595 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:19.599824 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:19.599838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.600933 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.600942 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:19.602752 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:19.602776 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.603559 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.603567 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:19.603810 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:19.604524 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:19.604552 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:19.608126 (+ 3574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260929 16:05:19.608763 29985 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:19.598549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:52615 (local address 127.20.121.190:45651)
0929 16:05:19.598735 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:19.598739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:19.598781 (+ 42us) server_negotiation.cc:407] Connection header received
0929 16:05:19.598849 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:19.598853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:19.598910 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:19.598991 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:19.600016 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.600765 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.602960 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.603375 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.603486 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:19.608358 (+ 4872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:52615: BlockingRecv error: recv got EOF from 127.20.121.129:52615 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 29 16:05:19 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:19 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:19.618607 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:19.609145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:50853)
0929 16:05:19.609521 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:19.609536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:19.609625 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:19.609987 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:19.609990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:19.610363 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:19.610525 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:19.610535 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.612005 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.612009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:19.612972 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:19.612983 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.614157 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.614163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:19.614329 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:19.615138 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:19.615156 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:19.618305 (+ 3149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259}
W20260929 16:05:19.618824 29985 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:19.609380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:50853 (local address 127.20.121.190:45651)
0929 16:05:19.609577 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:19.609583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:19.609604 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:05:19.609741 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:19.609747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:19.609839 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:19.609955 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:19.610679 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.611828 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.613162 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:19.613839 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:19.613979 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:19.618568 (+ 4589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:50853: BlockingRecv error: recv got EOF from 127.20.121.129:50853 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 16:05:19.618961 29983 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 16:05:19.935729 29987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:19.936048 29987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:19.936218 29987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:19.943897 29987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:19.944032 29987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:19.944208 29987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:05:19.954412 29987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.130.keytab
--principal=oryx/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:41687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=37903
--tserver_master_addrs=127.20.121.190:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:19.955868 29987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:19.957198 29987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:19.970801 29992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:19.970834 29993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:19.971060 29995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:19.971810 29987 server_base.cc:1034] running on GCE node
I20260929 16:05:19.972487 29987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:19.973483 29987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:19.974718 29987 hybrid_clock.cc:648] HybridClock initialized: now 1790697919974633 us; error 61 us; skew 500 ppm
I20260929 16:05:19.975082 29987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:19 dist-test-slave-x07d krb5kdc[29246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697919, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:19.978469 29987 init.cc:377] Logged in from keytab as oryx/127.20.121.130@KRBTEST.COM (short username oryx)
I20260929 16:05:19.980019 29987 webserver.cc:460] Webserver started at http://127.20.121.130:37903/ using document root <none> and password file <none>
I20260929 16:05:19.980701 29987 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:19.980810 29987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:19.986476 29987 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:05:19.989408 30002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:19.991061 29987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:05:19.991278 29987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "b62509bc6fb348099f9002bd04339a74"
format_stamp: "Formatted at 2026-09-29 16:05:17 on dist-test-slave-x07d"
server_key: "ca6d430cf608ce978c9785461381f824"
server_key_iv: "4e51724bf70f7ba592144b29eee864bb"
server_key_version: "encryptionkey@0"
I20260929 16:05:19.992354 29987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:20.020942 29987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:20.025710 29987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:20.026047 29987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:20.027241 29987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:20.028649 29987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:20.028766 29987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:20.028898 29987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:20.028991 29987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:20.065382 29987 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:41687
I20260929 16:05:20.065412 30114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:41687 every 8 connection(s)
I20260929 16:05:20.067274 29987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:05:20.071049 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 29987
I20260929 16:05:20.071923 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:43421
--local_ip_for_outbound_sockets=127.20.121.131
--tserver_master_addrs=127.20.121.190:45651
--webserver_port=45573
--webserver_interface=127.20.121.131
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.121.131
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
W20260929 16:05:20.087953 29985 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:20.070116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:33121 (local address 127.20.121.190:45651)
0929 16:05:20.070290 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:20.070295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:20.071994 (+ 1699us) server_negotiation.cc:407] Connection header received
0929 16:05:20.075447 (+ 3453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:20.075453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:20.075530 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:20.075633 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:20.078215 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.079069 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.080878 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.081766 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.081933 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.087596 (+ 5663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:33121: BlockingRecv error: recv got EOF from 127.20.121.130:33121 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 16:05:20.088495 30117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:20.070802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:33121)
0929 16:05:20.071783 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:20.071840 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:20.075036 (+ 3196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:20.075964 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:20.075980 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:20.076853 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:20.077807 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.077836 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.079279 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.079288 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.080688 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.080701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.082109 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.082116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.082336 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.083135 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:20.083165 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:20.087278 (+ 4113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":390,"threads_started":1}
W20260929 16:05:20.089267 30115 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:45651 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.121.190:45651: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:20.099124 30117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:20.089939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:59017)
0929 16:05:20.090163 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:20.090184 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:20.090313 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:20.090644 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:20.090648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:20.090999 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:20.091150 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.091159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.092216 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.092222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.093135 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.093147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.094266 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.094273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.094474 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.095271 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:20.095289 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:20.098783 (+ 3494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87}
W20260929 16:05:20.099231 29985 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:20.090098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:59017 (local address 127.20.121.190:45651)
0929 16:05:20.090285 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:20.090291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:20.090315 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:05:20.090427 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:20.090433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:20.090507 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:20.090620 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:20.091302 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.092065 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.093298 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.093742 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.094242 (+ 500us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.099004 (+ 4762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:59017: BlockingRecv error: recv got EOF from 127.20.121.130:59017 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:20.109565 30117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:20.100365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:55895)
0929 16:05:20.100650 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:20.100666 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:20.100762 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:20.101151 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:20.101154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:20.101468 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:20.101643 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.101651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.103320 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.103330 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.104270 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.104282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.105464 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.105469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.105621 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.106331 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:20.106349 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:20.109341 (+ 2992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151}
W20260929 16:05:20.109895 29985 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:20.100466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:55895 (local address 127.20.121.190:45651)
0929 16:05:20.100678 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:20.100685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:20.100710 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:05:20.100880 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:20.100886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:20.100972 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:20.101114 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:20.101811 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.103145 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.104425 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.105265 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.105438 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.109651 (+ 4213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:55895: BlockingRecv error: recv got EOF from 127.20.121.130:55895 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260929 16:05:20.109931 30115 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 16:05:20.464686 30118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:20.464998 30118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:20.465179 30118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:20.472779 30118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:20.472906 30118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:20.473109 30118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:05:20.483623 30118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/krb5kdc/oryx_127.20.121.131.keytab
--principal=oryx/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:43421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=45573
--tserver_master_addrs=127.20.121.190:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:20.485061 30118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:20.486467 30118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:20.501498 30123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:20.501415 30126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:20.501520 30124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:20.502429 30118 server_base.cc:1034] running on GCE node
I20260929 16:05:20.503213 30118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:20.504305 30118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:20.505594 30118 hybrid_clock.cc:648] HybridClock initialized: now 1790697920505496 us; error 68 us; skew 500 ppm
I20260929 16:05:20.505955 30118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697920, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:20.509400 30118 init.cc:377] Logged in from keytab as oryx/127.20.121.131@KRBTEST.COM (short username oryx)
I20260929 16:05:20.511070 30118 webserver.cc:460] Webserver started at http://127.20.121.131:45573/ using document root <none> and password file <none>
I20260929 16:05:20.511812 30118 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:20.511971 30118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:20.517765 30118 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260929 16:05:20.521433 30133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:20.523072 30118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:05:20.523275 30118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "3c9496aa17ec45c4af5b8a5232fc34ab"
format_stamp: "Formatted at 2026-09-29 16:05:17 on dist-test-slave-x07d"
server_key: "4ba741bacc183a5b88c3880d69125a66"
server_key_iv: "35cf873695481eb788e79909d5dd8e05"
server_key_version: "encryptionkey@0"
I20260929 16:05:20.524324 30118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:20.543092 30118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:20.547539 30118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:20.547842 30118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:20.548818 30118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:20.550252 30118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:20.550390 30118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:20.550550 30118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:20.550652 30118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:20.585238 30118 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:43421
I20260929 16:05:20.585325 30245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:43421 every 8 connection(s)
I20260929 16:05:20.586786 30118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:05:20.590575 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 30118
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
W20260929 16:05:20.605435 29985 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:20.589461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:33593 (local address 127.20.121.190:45651)
0929 16:05:20.589658 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:20.589662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:20.591129 (+ 1467us) server_negotiation.cc:407] Connection header received
0929 16:05:20.593860 (+ 2731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:20.593863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:20.593916 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:20.593992 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:20.595842 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.596674 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.599054 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.599448 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.599553 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.605101 (+ 5548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:33593: BlockingRecv error: recv got EOF from 127.20.121.131:33593 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 16:05:20.606274 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:20.589975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:33593)
0929 16:05:20.590932 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:20.590997 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:20.593518 (+ 2521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:20.594327 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:20.594343 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:20.594939 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:20.595664 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.595693 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.596861 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.596865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.598884 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.598903 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.599728 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.599750 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.600081 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.600893 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:20.600921 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:20.604903 (+ 3982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":319,"threads_started":1}
W20260929 16:05:20.607052 30246 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:45651 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.121.190:45651: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
I20260929 16:05:20.608389 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:20.591707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37486 (local address 127.20.121.190:45651)
0929 16:05:20.592299 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:20.592303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:20.592318 (+ 15us) server_negotiation.cc:407] Connection header received
0929 16:05:20.592368 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:20.592372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:20.592427 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:20.592519 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:20.594554 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.595283 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.596785 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.597336 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.597429 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.600655 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:20.600664 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:20.600683 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:20.600724 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:20.604239 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:20.605115 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:20.605129 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:20.605136 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:20.605340 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:20.605797 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:20.605805 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:20.605809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:20.606425 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:20.606772 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:20.607390 (+ 618us) server_negotiation.cc:299] Negotiation successful
0929 16:05:20.607885 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":385,"threads_started":1}
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:20.616689 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:20.607794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:58767)
0929 16:05:20.608016 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:20.608038 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:20.608166 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:20.608497 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:20.608505 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:20.608866 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:20.609028 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.609037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.610245 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.610252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.611527 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.611541 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.612378 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.612385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.612577 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.613261 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:20.613279 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:20.616478 (+ 3199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77}
W20260929 16:05:20.616972 29985 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:20.607858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:58767 (local address 127.20.121.190:45651)
0929 16:05:20.608015 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:20.608019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:20.608140 (+ 121us) server_negotiation.cc:407] Connection header received
0929 16:05:20.608288 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:20.608292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:20.608349 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:20.608427 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:20.609184 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.610090 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.611685 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.612193 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.612343 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.616708 (+ 4365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:58767: BlockingRecv error: recv got EOF from 127.20.121.131:58767 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:20.626650 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:20.617522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:53931)
0929 16:05:20.617697 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:20.617718 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:20.617841 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:20.618301 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:20.618306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:20.618639 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:20.618789 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.618797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.620234 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.620239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.621162 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.621173 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.621907 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.621913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.622195 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.622863 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:20.622880 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:20.626376 (+ 3496us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260929 16:05:20.627077 30246 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
W20260929 16:05:20.627424 29985 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:20.617693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:53931 (local address 127.20.121.190:45651)
0929 16:05:20.617958 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:20.617964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:20.617986 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:05:20.618086 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:20.618092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:20.618170 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:20.618272 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:20.618931 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.620083 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.621307 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.621752 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.621882 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.626687 (+ 4805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:53931: BlockingRecv error: recv got EOF from 127.20.121.131:53931 (error 108)
Metrics: {"server-negotiator.queue_time_us":150}
Sep 29 16:05:20 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:20.629359 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:20.619539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:37781)
0929 16:05:20.620030 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:20.620045 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:20.620193 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:20.620508 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:20.620512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:20.620954 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:20.621174 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.621184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.622243 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.622250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.623442 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:20.623460 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.624458 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.624466 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:20.624686 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.625816 (+ 1130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:20.625856 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:20.629082 (+ 3226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":252,"threads_started":1}
W20260929 16:05:20.629725 30250 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:20.619601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:37781 (local address 127.20.121.190:45651)
0929 16:05:20.619843 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:20.619850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:20.620138 (+ 288us) server_negotiation.cc:407] Connection header received
0929 16:05:20.620314 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:20.620320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:20.620383 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:20.620482 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:20.621314 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.622106 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.623629 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:20.624256 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:20.624400 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:20.629327 (+ 4927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:37781: BlockingRecv error: recv got EOF from 127.20.121.129:37781 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 16:05:21 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:21 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:21.120447 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:21.110763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:36025)
0929 16:05:21.111415 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:21.111432 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:21.111608 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:21.111968 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:21.111973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:21.112462 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:21.112650 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:21.112659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.113743 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.113749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:21.114727 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:21.114741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.115481 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.115488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:21.115657 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:21.116433 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:21.116456 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:21.120105 (+ 3649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":301,"threads_started":1}
W20260929 16:05:21.120577 30250 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:21.110838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:36025 (local address 127.20.121.190:45651)
0929 16:05:21.111191 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:21.111199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:21.111537 (+ 338us) server_negotiation.cc:407] Connection header received
0929 16:05:21.111753 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:21.111757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:21.111820 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:21.111938 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:21.112818 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.113589 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.114887 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.115287 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.115444 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:21.120319 (+ 4875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:36025: BlockingRecv error: recv got EOF from 127.20.121.130:36025 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Sep 29 16:05:21 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:21 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:21 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:21 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:21.647948 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:21.627923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:41561)
0929 16:05:21.628655 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:21.628676 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:21.628816 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:21.629388 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:21.629392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:21.629912 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:21.630258 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:21.630270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.631957 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.631963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:21.632977 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:21.632989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.633965 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.633971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:21.634190 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:21.644015 (+ 9825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:21.644040 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:21.647607 (+ 3567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":382,"threads_started":1}
I20260929 16:05:21.648454 30255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:21.630569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:42733)
0929 16:05:21.631337 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:21.631365 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:21.631545 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:21.632009 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:21.632019 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:21.632557 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:21.632783 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:21.632794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.634482 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.634487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:21.635742 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:21.635772 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.636718 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.636724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:21.637051 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:21.644037 (+ 6986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:21.644067 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:21.648136 (+ 4069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":362,"threads_started":1}
W20260929 16:05:21.648675 30253 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:21.627928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:41561 (local address 127.20.121.190:45651)
0929 16:05:21.628680 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:21.628688 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:21.628779 (+ 91us) server_negotiation.cc:407] Connection header received
0929 16:05:21.629089 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:21.629095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:21.629191 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:21.629319 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:21.630442 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.631793 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.633148 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.633746 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.633915 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:21.648335 (+ 14420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:41561: BlockingRecv error: recv got EOF from 127.20.121.131:41561 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":594,"thread_start_us":431,"threads_started":1}
W20260929 16:05:21.649250 30256 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:21.630632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:42733 (local address 127.20.121.190:45651)
0929 16:05:21.631672 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:21.631679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:21.631699 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:05:21.631759 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:21.631763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:21.631828 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:21.631966 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:21.633095 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.634302 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.636050 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:21.636512 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:21.636673 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:21.649033 (+ 12360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:42733: BlockingRecv error: recv got EOF from 127.20.121.129:42733 (error 108)
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":658,"threads_started":1}
Sep 29 16:05:22 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:22 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:22.132961 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:22.121653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:41697)
0929 16:05:22.122465 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:22.122489 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:22.122785 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:22.123270 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:22.123274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:22.123818 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:22.124040 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:22.124050 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.125375 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.125380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:22.126572 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:22.126589 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.127446 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.127452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:22.127761 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:22.128577 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:22.128600 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:22.132658 (+ 4058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":486,"threads_started":1}
W20260929 16:05:22.133380 30256 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:22.121808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:41697 (local address 127.20.121.190:45651)
0929 16:05:22.122088 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:22.122098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:22.122639 (+ 541us) server_negotiation.cc:407] Connection header received
0929 16:05:22.122958 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:22.122964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:22.123106 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:22.123202 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:22.124226 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.125174 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.126797 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.127225 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.127343 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:22.133035 (+ 5692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:41697: BlockingRecv error: recv got EOF from 127.20.121.130:41697 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 29 16:05:22 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:22 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:22 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:22 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:22.661311 30260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:22.649731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:40777)
0929 16:05:22.650496 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:22.650520 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:22.650667 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:22.651609 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:22.651615 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:22.652065 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:22.652324 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:22.652338 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.653933 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.653940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:22.655247 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:22.655269 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.656291 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.656297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:22.656506 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:22.657386 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:22.657417 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:22.661036 (+ 3619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":512,"threads_started":1}
W20260929 16:05:22.661898 30261 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:22.650125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:40777 (local address 127.20.121.190:45651)
0929 16:05:22.651191 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:22.651196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:22.651216 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:05:22.651276 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:22.651280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:22.651392 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:22.651518 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:22.652500 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.653676 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.655437 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.656121 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.656230 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:22.661492 (+ 5262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:40777: BlockingRecv error: recv got EOF from 127.20.121.129:40777 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":937,"spinlock_wait_cycles":3840,"thread_start_us":753,"threads_started":1}
I20260929 16:05:22.661942 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:22.649383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:43715)
0929 16:05:22.650100 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:22.650131 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:22.650364 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:22.651254 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:22.651259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:22.651916 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:22.652189 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:22.652203 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.653852 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.653858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:22.655445 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:22.655466 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.656540 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.656546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:22.656765 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:22.657598 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:22.657626 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:22.661597 (+ 3971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":380,"threads_started":1}
W20260929 16:05:22.662595 30259 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:22.649631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:43715 (local address 127.20.121.190:45651)
0929 16:05:22.650888 (+ 1257us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:22.650893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:22.650909 (+ 16us) server_negotiation.cc:407] Connection header received
0929 16:05:22.650978 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:22.650982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:22.651075 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:22.651218 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:22.652383 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.653663 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.655619 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:22.656321 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:22.656472 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:22.662254 (+ 5782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:43715: BlockingRecv error: recv got EOF from 127.20.121.131:43715 (error 108)
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":1128,"spinlock_wait_cycles":12672,"thread_start_us":983,"threads_started":1}
Sep 29 16:05:23 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:23 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:23.145531 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:23.134593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:56465)
0929 16:05:23.135515 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:23.135540 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:23.135790 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:23.136325 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:23.136331 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:23.136859 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:23.137110 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:23.137124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.138688 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.138692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:23.139736 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:23.139751 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.140601 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.140606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:23.140754 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:23.141560 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:23.141584 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:23.145190 (+ 3606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":593,"threads_started":1}
W20260929 16:05:23.146099 30259 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:23.134876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:56465 (local address 127.20.121.190:45651)
0929 16:05:23.135249 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:23.135258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:23.135778 (+ 520us) server_negotiation.cc:407] Connection header received
0929 16:05:23.135953 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:23.135959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:23.136106 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:23.136263 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:23.137341 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.138512 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.139938 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.140407 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.140549 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:23.145744 (+ 5195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:56465: BlockingRecv error: recv got EOF from 127.20.121.130:56465 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Sep 29 16:05:23 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:23 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:23 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:23 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:23.676705 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:23.662912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:48043)
0929 16:05:23.663717 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:23.663746 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:23.663983 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:23.664964 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:23.664971 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:23.665564 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:23.665816 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:23.665831 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.667587 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.667593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:23.668668 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:23.668687 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.669591 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.669597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:23.669831 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:23.670916 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:23.670945 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:23.676268 (+ 5323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":388,"threads_started":1}
I20260929 16:05:23.676937 30265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:23.663502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:34349)
0929 16:05:23.665399 (+ 1897us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:23.665420 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:23.665618 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:23.666323 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:23.666329 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:23.666886 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:23.667147 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:23.667162 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.668504 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.668511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:23.669488 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:23.669505 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.670776 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.670788 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:23.671027 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:23.671855 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:23.671889 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:23.676484 (+ 4595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1770,"thread_start_us":1396,"threads_started":1}
W20260929 16:05:23.677013 30264 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:23.663209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:48043 (local address 127.20.121.190:45651)
0929 16:05:23.664122 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:23.664132 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:23.664165 (+ 33us) server_negotiation.cc:407] Connection header received
0929 16:05:23.664264 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:23.664274 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:23.664419 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:23.664617 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:23.666061 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.667381 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.668919 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.669405 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.669559 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:23.676585 (+ 7026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:48043: BlockingRecv error: recv got EOF from 127.20.121.129:48043 (error 108)
Metrics: {"server-negotiator.queue_time_us":720,"spinlock_wait_cycles":15744,"thread_start_us":676,"threads_started":1}
W20260929 16:05:23.677255 30266 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:23.663764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:34349 (local address 127.20.121.190:45651)
0929 16:05:23.664724 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:23.664730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:23.665574 (+ 844us) server_negotiation.cc:407] Connection header received
0929 16:05:23.665967 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:23.665974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:23.666110 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:23.666234 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:23.667374 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.668302 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.669689 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:23.670509 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:23.670677 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:23.676802 (+ 6125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:34349: BlockingRecv error: recv got EOF from 127.20.121.131:34349 (error 108)
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":592,"threads_started":1}
Sep 29 16:05:24 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:24 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:24.161196 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:24.147031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:45053)
0929 16:05:24.147795 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:24.147826 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:24.148065 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:24.148652 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:24.148660 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:24.149317 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:24.149605 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:24.149623 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.151378 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.151385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:24.153855 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:24.153881 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.155034 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.155042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:24.155298 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:24.156720 (+ 1422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:24.156751 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:24.160877 (+ 4126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":430,"threads_started":1}
W20260929 16:05:24.161528 30266 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:24.147175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:45053 (local address 127.20.121.190:45651)
0929 16:05:24.147594 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:24.147601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:24.147987 (+ 386us) server_negotiation.cc:407] Connection header received
0929 16:05:24.148258 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:24.148265 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:24.148377 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:24.148547 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:24.149812 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.151073 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.154110 (+ 3037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.154784 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.154965 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:24.161215 (+ 6250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:45053: BlockingRecv error: recv got EOF from 127.20.121.130:45053 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Sep 29 16:05:24 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:24 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:24 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:24 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:24.694242 30269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:24.678499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:41003)
0929 16:05:24.682711 (+ 4212us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:24.682744 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:24.682971 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:24.683958 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:24.683962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:24.684579 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:24.684855 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:24.684870 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.686460 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.686466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:24.687703 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:24.687716 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.688856 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.688860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:24.689057 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:24.689884 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:24.689909 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:24.693831 (+ 3922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":3978,"thread_start_us":3771,"threads_started":1}
W20260929 16:05:24.694533 30270 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:24.678723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:41003 (local address 127.20.121.190:45651)
0929 16:05:24.683540 (+ 4817us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:24.683547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:24.683570 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:05:24.683639 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:24.683644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:24.683742 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:24.683867 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:24.685047 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.686259 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.687891 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.688656 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.688827 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:24.694236 (+ 5409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:41003: BlockingRecv error: recv got EOF from 127.20.121.129:41003 (error 108)
Metrics: {"server-negotiator.queue_time_us":4680,"thread_start_us":4497,"threads_started":1}
I20260929 16:05:24.694913 30268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:24.678527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:44167)
0929 16:05:24.683125 (+ 4598us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:24.683160 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:24.683401 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:24.684556 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:24.684562 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:24.685152 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:24.685371 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:24.685383 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.687258 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.687263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:24.688920 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:24.688940 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.689946 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.689952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:24.690192 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:24.690988 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:24.691007 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:24.694549 (+ 3542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4435,"thread_start_us":4311,"threads_started":1}
W20260929 16:05:24.695398 30271 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:24.678673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:44167 (local address 127.20.121.190:45651)
0929 16:05:24.682328 (+ 3655us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:24.682335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:24.683336 (+ 1001us) server_negotiation.cc:407] Connection header received
0929 16:05:24.683638 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:24.683642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:24.683721 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:24.684486 (+ 765us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:24.685594 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.687046 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.689128 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:24.689734 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:24.689914 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:24.695154 (+ 5240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:44167: BlockingRecv error: recv got EOF from 127.20.121.131:44167 (error 108)
Metrics: {"server-negotiator.queue_time_us":3488,"thread_start_us":3369,"threads_started":1}
Sep 29 16:05:25 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:25 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:25.172371 30272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:25.162878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:59447)
0929 16:05:25.163518 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:25.163537 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:25.163682 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:25.164077 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:25.164080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:25.164528 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:25.164771 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:25.164781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.166001 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.166005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:25.167080 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:25.167095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.167893 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.167897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:25.168032 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:25.168803 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:25.168824 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:25.172062 (+ 3238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":336,"threads_started":1}
W20260929 16:05:25.172595 30271 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:25.163193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:59447 (local address 127.20.121.190:45651)
0929 16:05:25.163385 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:25.163390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:25.163631 (+ 241us) server_negotiation.cc:407] Connection header received
0929 16:05:25.163785 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:25.163789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:25.163858 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:25.164029 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:25.164976 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.165834 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.167244 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.167721 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.167838 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:25.172292 (+ 4454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:59447: BlockingRecv error: recv got EOF from 127.20.121.130:59447 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 29 16:05:25 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:25 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:25 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:25 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:25.711943 30273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:25.695984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:45581)
0929 16:05:25.697975 (+ 1991us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:25.697998 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:25.698212 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:25.699194 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:25.699200 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:25.699818 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:25.700054 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:25.700071 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.702179 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.702186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:25.703926 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:25.703952 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.705481 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.705491 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:25.705709 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:25.706775 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:25.706804 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:25.711457 (+ 4653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1852,"thread_start_us":1435,"threads_started":1}
I20260929 16:05:25.712579 30274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:25.696156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:55303)
0929 16:05:25.697023 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:25.697058 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:25.697850 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:25.699318 (+ 1468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:25.699326 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:25.699885 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:25.700116 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:25.700132 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.702179 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.702186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:25.703925 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:25.703952 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.705506 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.705522 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:25.705726 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:25.706774 (+ 1048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:25.706804 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:25.712161 (+ 5357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":413,"threads_started":1}
W20260929 16:05:25.712880 30275 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:25.696154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:45581 (local address 127.20.121.190:45651)
0929 16:05:25.697997 (+ 1843us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:25.698033 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:25.698137 (+ 104us) server_negotiation.cc:407] Connection header received
0929 16:05:25.698582 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:25.698591 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:25.698898 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:25.699086 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:25.700255 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.701924 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.704330 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.705173 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.705361 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:25.711782 (+ 6421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:45581: BlockingRecv error: recv got EOF from 127.20.121.129:45581 (error 108)
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":1699,"spinlock_wait_cycles":896,"thread_start_us":1535,"threads_started":1}
W20260929 16:05:25.713322 30276 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:25.697902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:55303 (local address 127.20.121.190:45651)
0929 16:05:25.698723 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:25.698729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:25.698749 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:05:25.698797 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:25.698804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:25.698980 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:25.699864 (+ 884us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:25.700464 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.701924 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.704330 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:25.705173 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:25.705360 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:25.712971 (+ 7611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:55303: BlockingRecv error: recv got EOF from 127.20.121.131:55303 (error 108)
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":628,"spinlock_wait_cycles":3840,"thread_start_us":324,"threads_started":1}
Sep 29 16:05:26 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:26 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:26.184039 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:26.173714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:56473)
0929 16:05:26.174320 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:26.174337 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:26.174475 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:26.175051 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:26.175056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:26.175482 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:26.175660 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:26.175669 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.176878 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.176882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:26.177829 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:26.177842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.178794 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.178799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:26.179000 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:26.179786 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:26.179805 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:26.183783 (+ 3978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":362,"threads_started":1}
W20260929 16:05:26.184465 30276 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:26.173869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:56473 (local address 127.20.121.190:45651)
0929 16:05:26.174165 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:26.174170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:26.174496 (+ 326us) server_negotiation.cc:407] Connection header received
0929 16:05:26.174598 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:26.174603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:26.174744 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:26.174907 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:26.175868 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.176689 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.178134 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.178619 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.178775 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:26.184091 (+ 5316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:56473: BlockingRecv error: recv got EOF from 127.20.121.130:56473 (error 108)
Metrics: {"server-negotiator.queue_time_us":181}
Sep 29 16:05:26 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:26 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:26 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:26 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:26.739398 30280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:26.713742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:54333)
0929 16:05:26.720535 (+ 6793us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:26.720561 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:26.720747 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:26.722035 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:26.722042 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:26.722682 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:26.722902 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:26.722912 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.724276 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.724282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:26.725306 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:26.725319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.726205 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.726212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:26.726422 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:26.734597 (+ 8175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:26.734621 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:26.739011 (+ 4390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":6660,"thread_start_us":6511,"threads_started":1}
I20260929 16:05:26.739451 30278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:26.713556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:36373)
0929 16:05:26.721732 (+ 8176us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:26.721752 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:26.721927 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:26.722488 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:26.722491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:26.723083 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:26.723326 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:26.723338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.724658 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.724664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:26.725809 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:26.725821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.726961 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.726969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:26.727162 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:26.734531 (+ 7369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:26.734606 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:26.739014 (+ 4408us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":8051,"thread_start_us":7817,"threads_started":1}
W20260929 16:05:26.739653 30279 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:26.713806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:54333 (local address 127.20.121.190:45651)
0929 16:05:26.721572 (+ 7766us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:26.721577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:26.721597 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:05:26.721652 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:26.721657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:26.721734 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:26.721879 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:26.723141 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.724082 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.725459 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.725967 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.726169 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:26.739244 (+ 13075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:54333: BlockingRecv error: recv got EOF from 127.20.121.131:54333 (error 108)
Metrics: {"server-negotiator.queue_time_us":7669,"thread_start_us":7045,"threads_started":1}
W20260929 16:05:26.739758 30281 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:26.713633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:36373 (local address 127.20.121.190:45651)
0929 16:05:26.721104 (+ 7471us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:26.721109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:26.722106 (+ 997us) server_negotiation.cc:407] Connection header received
0929 16:05:26.722156 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:26.722160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:26.722218 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:26.722308 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:26.723541 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.724493 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.725976 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:26.726791 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:26.726936 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:26.739305 (+ 12369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:36373: BlockingRecv error: recv got EOF from 127.20.121.129:36373 (error 108)
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":7321,"thread_start_us":7674,"threads_started":1}
Sep 29 16:05:27 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:27 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:27.193843 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:27.185002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:58815)
0929 16:05:27.185524 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:27.185541 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:27.185692 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:27.185971 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:27.185974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:27.186493 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:27.186697 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:27.186707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.187806 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.187810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:27.188771 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:27.188784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.189510 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.189515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:27.189639 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:27.190384 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:27.190406 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:27.193512 (+ 3106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":251,"threads_started":1}
W20260929 16:05:27.194119 30281 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:27.185048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:58815 (local address 127.20.121.190:45651)
0929 16:05:27.185329 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:27.185339 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:27.185644 (+ 305us) server_negotiation.cc:407] Connection header received
0929 16:05:27.185791 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:27.185795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:27.185854 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:27.185937 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:27.186835 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.187629 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.188926 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.189337 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.189448 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:27.193796 (+ 4348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:58815: BlockingRecv error: recv got EOF from 127.20.121.130:58815 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Sep 29 16:05:27 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:27 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:27 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:27 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:27.752447 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:27.740945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:57401)
0929 16:05:27.742667 (+ 1722us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:27.742692 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:27.742815 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:27.743384 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:27.743389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:27.743950 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:27.744225 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:27.744243 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.745997 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.746003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:27.746971 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:27.746983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.747765 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.747770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:27.747938 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:27.748592 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:27.748611 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:27.752017 (+ 3406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1545,"thread_start_us":1197,"threads_started":1}
W20260929 16:05:27.752799 30286 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:27.742049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:57401 (local address 127.20.121.190:45651)
0929 16:05:27.742935 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:27.742944 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:27.742968 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:05:27.743015 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:27.743020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:27.743084 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:27.743270 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:27.744491 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.745834 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.747130 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.747593 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.747747 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:27.752366 (+ 4619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:57401: BlockingRecv error: recv got EOF from 127.20.121.131:57401 (error 108)
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":765,"spinlock_wait_cycles":403840,"thread_start_us":616,"threads_started":1}
I20260929 16:05:27.753425 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:27.740946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:36497)
0929 16:05:27.741766 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:27.741797 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:27.741971 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:27.743414 (+ 1443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:27.743420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:27.744102 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:27.744397 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:27.744414 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.746231 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.746237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:27.747348 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:27.747361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.748308 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.748315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:27.748535 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:27.749268 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:27.749287 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:27.753169 (+ 3882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":344,"threads_started":1}
W20260929 16:05:27.753638 30285 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:27.741062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:36497 (local address 127.20.121.190:45651)
0929 16:05:27.742246 (+ 1184us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:27.742258 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:27.742292 (+ 34us) server_negotiation.cc:407] Connection header received
0929 16:05:27.742381 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:27.742387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:27.742521 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:27.743270 (+ 749us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:27.744565 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.746002 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.747531 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:27.748080 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:27.748238 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:27.753328 (+ 5090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:36497: BlockingRecv error: recv got EOF from 127.20.121.129:36497 (error 108)
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":969,"thread_start_us":758,"threads_started":1}
Sep 29 16:05:28 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:28 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:28.206626 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:28.195069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:54845)
0929 16:05:28.195809 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:28.195830 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:28.196044 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:28.196468 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:28.196474 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:28.197010 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:28.197199 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:28.197209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.198757 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.198766 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:28.199887 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:28.199901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.200903 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.200913 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:28.201141 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:28.202427 (+ 1286us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:28.202450 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:28.206325 (+ 3875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":387,"threads_started":1}
W20260929 16:05:28.206844 30285 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:28.195267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:54845 (local address 127.20.121.190:45651)
0929 16:05:28.195590 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:28.195599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:28.195938 (+ 339us) server_negotiation.cc:407] Connection header received
0929 16:05:28.196206 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:28.196214 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:28.196308 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:28.196433 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:28.197359 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.198579 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.200080 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.200651 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.200817 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:28.206614 (+ 5797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:54845: BlockingRecv error: recv got EOF from 127.20.121.130:54845 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Sep 29 16:05:28 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:28 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:28 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:28 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:28.768393 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:28.753971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:60129)
0929 16:05:28.755543 (+ 1572us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:28.755562 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:28.755715 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:28.756565 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:28.756572 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:28.757301 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:28.757525 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:28.757537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.759472 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.759479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:28.761276 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:28.761300 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.762635 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.762644 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:28.762840 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:28.763495 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:28.763525 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:28.767988 (+ 4463us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1417,"thread_start_us":1224,"threads_started":1}
I20260929 16:05:28.768410 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:28.754588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:37293)
0929 16:05:28.755332 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:28.755359 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:28.755550 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:28.756489 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:28.756494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:28.757091 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:28.757331 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:28.757345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.759159 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.759164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:28.760346 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:28.760359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.761854 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.761863 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:28.762137 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:28.763277 (+ 1140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:28.763308 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:28.767988 (+ 4680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":559,"threads_started":1}
W20260929 16:05:28.769137 30289 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:28.754160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:60129 (local address 127.20.121.190:45651)
0929 16:05:28.754823 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:28.754831 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:28.755653 (+ 822us) server_negotiation.cc:407] Connection header received
0929 16:05:28.755915 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:28.755920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:28.756079 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:28.756200 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:28.757822 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.759267 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.761578 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.762403 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.762536 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:28.768375 (+ 5839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:60129: BlockingRecv error: recv got EOF from 127.20.121.131:60129 (error 108)
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":500,"thread_start_us":320,"threads_started":1}
W20260929 16:05:28.769625 30291 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:28.755000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:37293 (local address 127.20.121.190:45651)
0929 16:05:28.755891 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:28.755899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:28.755923 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:05:28.755999 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:28.756005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:28.756092 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:28.756206 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:28.757528 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.758995 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.760725 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:28.761578 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:28.761736 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:28.768366 (+ 6630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:37293: BlockingRecv error: recv got EOF from 127.20.121.129:37293 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":724,"spinlock_wait_cycles":8064,"thread_start_us":562,"threads_started":1}
Sep 29 16:05:29 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:29 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
W20260929 16:05:29.219980 30291 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:29.208149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:55947 (local address 127.20.121.190:45651)
0929 16:05:29.208393 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:29.208398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:29.208840 (+ 442us) server_negotiation.cc:407] Connection header received
0929 16:05:29.209001 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:29.209006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:29.209090 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:29.209224 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:29.210248 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.211190 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.212676 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.213219 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.213356 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:29.219718 (+ 6362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:55947: BlockingRecv error: recv got EOF from 127.20.121.130:55947 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260929 16:05:29.220288 30292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:29.208104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:55947)
0929 16:05:29.208707 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:29.208724 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:29.208860 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:29.209355 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:29.209359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:29.209761 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:29.209957 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:29.209967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.211375 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.211380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:29.212459 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:29.212476 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.213413 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.213417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:29.213555 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:29.214428 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:29.214453 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:29.219441 (+ 4988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":318,"threads_started":1}
Sep 29 16:05:29 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:29 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:29 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:29 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:29.781643 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:29.769786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:57983)
0929 16:05:29.770507 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:29.770532 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:29.770671 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:29.771638 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:29.771643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:29.772197 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:29.772373 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:29.772383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.773824 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.773830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:29.775042 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:29.775056 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.776161 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.776169 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:29.776399 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:29.777380 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:29.777402 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:29.781380 (+ 3978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":415,"threads_started":1}
I20260929 16:05:29.782423 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:29.770246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:52333)
0929 16:05:29.770957 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:29.770981 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:29.771197 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:29.771915 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:29.771920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:29.772427 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:29.772742 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:29.772756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.774405 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.774410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:29.775915 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:29.775932 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.777093 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.777100 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:29.777360 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:29.778363 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:29.778388 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:29.782110 (+ 3722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":597,"threads_started":1}
W20260929 16:05:29.782485 30294 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:29.770076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:57983 (local address 127.20.121.190:45651)
0929 16:05:29.771256 (+ 1180us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:29.771265 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:29.771286 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:05:29.771365 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:29.771370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:29.771435 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:29.771545 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:29.772584 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.773612 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.775219 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.775934 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.776094 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:29.782169 (+ 6075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:57983: BlockingRecv error: recv got EOF from 127.20.121.129:57983 (error 108)
Metrics: {"server-negotiator.queue_time_us":1024,"spinlock_wait_cycles":3328,"thread_start_us":827,"threads_started":1}
W20260929 16:05:29.782588 30296 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:29.770675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:52333 (local address 127.20.121.190:45651)
0929 16:05:29.771452 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:29.771457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:29.771473 (+ 16us) server_negotiation.cc:407] Connection header received
0929 16:05:29.771557 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:29.771563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:29.771657 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:29.771790 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:29.772987 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.774236 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.776115 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:29.776896 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:29.777041 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:29.782303 (+ 5262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:52333: BlockingRecv error: recv got EOF from 127.20.121.131:52333 (error 108)
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":463,"threads_started":1}
Sep 29 16:05:30 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:30 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:30.233253 30297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:30.221792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:54503)
0929 16:05:30.222472 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:30.222493 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:30.222641 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:30.223165 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:30.223169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:30.223567 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:30.223803 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:30.223816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.225341 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.225347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:30.226794 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:30.226812 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.227649 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.227654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:30.227780 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:30.228720 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:30.228740 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:30.232843 (+ 4103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":405,"threads_started":1}
W20260929 16:05:30.233390 30296 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:30.221961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:54503 (local address 127.20.121.190:45651)
0929 16:05:30.222269 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:30.222275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:30.222598 (+ 323us) server_negotiation.cc:407] Connection header received
0929 16:05:30.222782 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:30.222788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:30.222868 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:30.223005 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:30.224007 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.225098 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.226954 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.227414 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.227524 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:30.233105 (+ 5581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:54503: BlockingRecv error: recv got EOF from 127.20.121.130:54503 (error 108)
Metrics: {"server-negotiator.queue_time_us":150}
Sep 29 16:05:30 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:30 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:30 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:30.793895 30298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:30.783007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:59381)
0929 16:05:30.783631 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:30.783656 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:30.783830 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:30.784362 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:30.784371 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:30.784975 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:30.785234 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:30.785244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.786468 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.786474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:30.787419 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:30.787430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.788168 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.788174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:30.788352 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:30.789159 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:30.789196 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:30.793527 (+ 4331us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":780,"threads_started":1}
W20260929 16:05:30.794094 30299 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:30.783368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:59381 (local address 127.20.121.190:45651)
0929 16:05:30.783904 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:30.783911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:30.783939 (+ 28us) server_negotiation.cc:407] Connection header received
0929 16:05:30.784021 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:30.784029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:30.784137 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:30.784298 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:30.785380 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.786310 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.787575 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.788003 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.788145 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:30.793711 (+ 5566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:59381: BlockingRecv error: recv got EOF from 127.20.121.129:59381 (error 108)
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":429,"threads_started":1}
Sep 29 16:05:30 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:30.795598 30300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:30.783658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:45999)
0929 16:05:30.784875 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:30.784898 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:30.785031 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:30.785544 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:30.785550 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:30.786153 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:30.786361 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:30.786371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.787669 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.787678 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:30.789082 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:30.789102 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.790115 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.790132 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:30.790339 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:30.791274 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:30.791295 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:30.795196 (+ 3901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1087,"thread_start_us":968,"threads_started":1}
W20260929 16:05:30.795678 30301 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:30.783728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:45999 (local address 127.20.121.190:45651)
0929 16:05:30.784575 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:30.784585 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:30.784999 (+ 414us) server_negotiation.cc:407] Connection header received
0929 16:05:30.785219 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:30.785224 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:30.785313 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:30.785525 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:30.786553 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.787502 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.789273 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:30.789854 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:30.790046 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:30.795386 (+ 5340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:45999: BlockingRecv error: recv got EOF from 127.20.121.131:45999 (error 108)
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":445,"threads_started":1}
Sep 29 16:05:31 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:31 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:31.245334 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:31.234771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:40919)
0929 16:05:31.235587 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:31.235615 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:31.235801 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:31.236245 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:31.236250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:31.236771 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:31.236990 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:31.237000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.238197 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.238203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:31.239320 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:31.239336 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.240124 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.240129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:31.240293 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:31.241206 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:31.241226 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:31.245018 (+ 3792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":435,"threads_started":1}
W20260929 16:05:31.245510 30301 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:31.235004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:40919 (local address 127.20.121.190:45651)
0929 16:05:31.235186 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:31.235191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:31.235743 (+ 552us) server_negotiation.cc:407] Connection header received
0929 16:05:31.235946 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:31.235950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:31.236024 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:31.236130 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:31.237163 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.237960 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.239498 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.239930 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.240039 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:31.245266 (+ 5227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:40919: BlockingRecv error: recv got EOF from 127.20.121.130:40919 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 29 16:05:31 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:31 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:31 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:31 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:31.816233 30303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:31.795200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:45215)
0929 16:05:31.795987 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:31.796015 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:31.796201 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:31.796740 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:31.796746 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:31.797469 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:31.797682 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:31.797735 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.799361 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.799370 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:31.800953 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:31.800995 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.802065 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.802073 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:31.802429 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:31.811181 (+ 8752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:31.811220 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:31.815881 (+ 4661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":465,"threads_started":1}
W20260929 16:05:31.816507 30304 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:31.795301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:45215 (local address 127.20.121.190:45651)
0929 16:05:31.796122 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:31.796130 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:31.796151 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:05:31.796431 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:31.796436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:31.796539 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:31.796680 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:31.797952 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.799192 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.801143 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.801831 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.801979 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:31.816190 (+ 14211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:45215: BlockingRecv error: recv got EOF from 127.20.121.129:45215 (error 108)
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":504,"threads_started":1}
I20260929 16:05:31.816641 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:31.796639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:48887)
0929 16:05:31.797548 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:31.797577 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:31.797697 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:31.798188 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:31.798195 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:31.798737 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:31.798960 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:31.798984 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.800868 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.800876 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:31.802349 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:31.802362 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.803333 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.803339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:31.803528 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:31.811181 (+ 7653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:31.811220 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:31.816302 (+ 5082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":630,"threads_started":1}
W20260929 16:05:31.817216 30306 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:31.796704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:48887 (local address 127.20.121.190:45651)
0929 16:05:31.797321 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:31.797328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:31.797685 (+ 357us) server_negotiation.cc:407] Connection header received
0929 16:05:31.797863 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:31.797872 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:31.797954 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:31.798119 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:31.799237 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.800724 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.802493 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:31.803108 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:31.803248 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:31.816990 (+ 13742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:48887: BlockingRecv error: recv got EOF from 127.20.121.131:48887 (error 108)
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":342,"threads_started":1}
Sep 29 16:05:32 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:32 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:32.256868 30307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:32.246677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:49229)
0929 16:05:32.247411 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:32.247436 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:32.247616 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:32.248186 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:32.248191 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:32.248792 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:32.249035 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:32.249052 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.250267 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.250272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:32.251322 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:32.251337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.252069 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.252074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:32.252235 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:32.252965 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:32.252987 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:32.256545 (+ 3558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":394,"threads_started":1}
W20260929 16:05:32.257159 30306 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:32.246895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:49229 (local address 127.20.121.190:45651)
0929 16:05:32.247208 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:32.247214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:32.247550 (+ 336us) server_negotiation.cc:407] Connection header received
0929 16:05:32.247751 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:32.247757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:32.247857 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:32.247990 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:32.249212 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.250102 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.251481 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.251874 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.251988 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:32.256779 (+ 4791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:49229: BlockingRecv error: recv got EOF from 127.20.121.130:49229 (error 108)
Metrics: {"server-negotiator.queue_time_us":184}
Sep 29 16:05:32 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:32 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:32 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:32 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:32.842532 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:32.817634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:48855)
0929 16:05:32.830888 (+ 13254us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:32.830921 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:32.831193 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:32.831818 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:32.831825 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:32.832544 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:32.832746 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:32.832757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.834062 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.834069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:32.835527 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:32.835543 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.836608 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.836615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:32.836873 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:32.837883 (+ 1010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:32.837906 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:32.842192 (+ 4286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13087,"thread_start_us":12904,"threads_started":1}
W20260929 16:05:32.842923 30309 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:32.817707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:48855 (local address 127.20.121.190:45651)
0929 16:05:32.830476 (+ 12769us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:32.830485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:32.831343 (+ 858us) server_negotiation.cc:407] Connection header received
0929 16:05:32.831449 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:32.831457 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:32.831572 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:32.831738 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:32.832920 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.833855 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.835702 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.836370 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.836523 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:32.842529 (+ 6006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:48855: BlockingRecv error: recv got EOF from 127.20.121.129:48855 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":12601,"thread_start_us":12453,"threads_started":1}
I20260929 16:05:32.843127 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:32.817751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:53687)
0929 16:05:32.831324 (+ 13573us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:32.831349 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:32.831506 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:32.832379 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:32.832387 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:32.832946 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:32.833164 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:32.833176 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.834857 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.834861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:32.836123 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:32.836138 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.837522 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.837531 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:32.837783 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:32.838695 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:32.838722 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:32.842574 (+ 3852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":13413,"thread_start_us":13287,"threads_started":1}
W20260929 16:05:32.843336 30311 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:32.817764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:53687 (local address 127.20.121.190:45651)
0929 16:05:32.831924 (+ 14160us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:32.831936 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:32.831956 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:05:32.832018 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:32.832024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:32.832107 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:32.832258 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:32.833381 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.834685 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.836316 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:32.837302 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:32.837457 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:32.842936 (+ 5479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:53687: BlockingRecv error: recv got EOF from 127.20.121.131:53687 (error 108)
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":14002,"thread_start_us":13797,"threads_started":1}
Sep 29 16:05:33 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:33 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:33.267886 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:33.258290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:58015)
0929 16:05:33.258960 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:33.258979 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:33.259122 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:33.259475 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:33.259479 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:33.259981 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:33.260191 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:33.260202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.261226 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.261231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:33.262300 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:33.262314 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.263029 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.263033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:33.263160 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:33.263966 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:33.263986 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:33.267554 (+ 3568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":353,"threads_started":1}
W20260929 16:05:33.268483 30311 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:33.258405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:58015 (local address 127.20.121.190:45651)
0929 16:05:33.258598 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:33.258604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:33.259100 (+ 496us) server_negotiation.cc:407] Connection header received
0929 16:05:33.259241 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:33.259245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:33.259323 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:33.259417 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:33.260338 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.261083 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.262445 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.262843 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.262943 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:33.267997 (+ 5054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:58015: BlockingRecv error: recv got EOF from 127.20.121.130:58015 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 29 16:05:33 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:33 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:33 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:33 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:33.854419 30313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:33.843799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:44185)
0929 16:05:33.844430 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:33.844458 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:33.844707 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:33.845444 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:33.845450 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:33.845869 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:33.846145 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:33.846157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.847339 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.847346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:33.848407 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:33.848419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.849223 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.849229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:33.849433 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:33.850246 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:33.850291 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:33.853967 (+ 3676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":322,"threads_started":1}
W20260929 16:05:33.854910 30314 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:33.843952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:44185 (local address 127.20.121.190:45651)
0929 16:05:33.844771 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:33.844777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:33.844802 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:05:33.844867 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:33.844875 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:33.844969 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:33.845153 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:33.846308 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.847157 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.848572 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.849049 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.849208 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:33.854294 (+ 5086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:44185: BlockingRecv error: recv got EOF from 127.20.121.129:44185 (error 108)
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":553,"threads_started":1}
I20260929 16:05:33.855352 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:33.844282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:39373)
0929 16:05:33.845303 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:33.845327 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:33.845467 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:33.845926 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:33.845932 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:33.846442 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:33.846634 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:33.846644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.847922 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.847928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:33.848916 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:33.848928 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.849791 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.849797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:33.849982 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:33.851119 (+ 1137us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:33.851138 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:33.854954 (+ 3816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":745,"threads_started":1}
W20260929 16:05:33.855520 30316 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:33.844700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:39373 (local address 127.20.121.190:45651)
0929 16:05:33.845640 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:33.845645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:33.845659 (+ 14us) server_negotiation.cc:407] Connection header received
0929 16:05:33.845706 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:33.845710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:33.845764 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:33.845890 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:33.846819 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.847756 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.849083 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:33.849586 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:33.849737 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:33.855209 (+ 5472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:39373: BlockingRecv error: recv got EOF from 127.20.121.131:39373 (error 108)
Metrics: {"server-negotiator.queue_time_us":826,"thread_start_us":610,"threads_started":1}
Sep 29 16:05:34 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:34 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:34.279386 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:34.269053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:39899)
0929 16:05:34.269766 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:34.269786 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:34.270044 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:34.270441 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:34.270446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:34.270924 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:34.271140 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:34.271150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.272389 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.272395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:34.273384 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:34.273396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.274230 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.274236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:34.274431 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:34.275318 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:34.275341 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:34.279035 (+ 3694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":371,"threads_started":1}
W20260929 16:05:34.279798 30316 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:34.269266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:39899 (local address 127.20.121.190:45651)
0929 16:05:34.269579 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:34.269587 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:34.269914 (+ 327us) server_negotiation.cc:407] Connection header received
0929 16:05:34.270197 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:34.270205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:34.270284 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:34.270401 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:34.271335 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.272221 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.273544 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.273993 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.274204 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:34.279480 (+ 5276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:39899: BlockingRecv error: recv got EOF from 127.20.121.130:39899 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Sep 29 16:05:34 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:34 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:34 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.129@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:34 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.131@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:34.868566 30318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:34.855922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.129:48067)
0929 16:05:34.856703 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:34.856736 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:34.856975 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:34.858086 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:34.858092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:34.858541 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:34.858807 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:34.858830 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.859994 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.860000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:34.861298 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:34.861318 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.862377 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.862388 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:34.862633 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:34.863558 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:34.863594 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:34.867911 (+ 4317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":331,"threads_started":1}
W20260929 16:05:34.868556 30319 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:34.856239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:48067 (local address 127.20.121.190:45651)
0929 16:05:34.857640 (+ 1401us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:34.857647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:34.857666 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:05:34.857733 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:34.857738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:34.857818 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:34.858816 (+ 998us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:34.858937 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.859786 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.861514 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.862106 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.862378 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:34.868184 (+ 5806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.129:48067: BlockingRecv error: recv got EOF from 127.20.121.129:48067 (error 108)
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":1242,"thread_start_us":1066,"threads_started":1}
I20260929 16:05:34.868973 30320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:34.856520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.131:49373)
0929 16:05:34.857825 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:34.857850 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:34.857986 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:34.858783 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:34.858788 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:34.859213 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:34.859423 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:34.859433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.860738 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.860746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:34.861974 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:34.861987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.863307 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.863313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:34.863540 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:34.864388 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:34.864414 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:34.868538 (+ 4124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1175,"thread_start_us":1037,"threads_started":1}
W20260929 16:05:34.869091 30321 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:34.856731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:49373 (local address 127.20.121.190:45651)
0929 16:05:34.858412 (+ 1681us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:34.858417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:34.858436 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:05:34.858492 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:34.858496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:34.858558 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:34.858672 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:34.859636 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.860548 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.862202 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:34.863129 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:34.863283 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:34.868855 (+ 5572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.131:49373: BlockingRecv error: recv got EOF from 127.20.121.131:49373 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1567,"thread_start_us":1404,"threads_started":1}
Sep 29 16:05:35 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
Sep 29 16:05:35 dist-test-slave-x07d krb5kdc[29246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.121.130@KRBTEST.COM for oryx/127.20.121.190@KRBTEST.COM, Server not found in Kerberos database
I20260929 16:05:35.291083 30322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:35.281140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45651 (local address 127.20.121.130:35181)
0929 16:05:35.281857 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:35.281875 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:35.282079 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:35.282607 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:35.282612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:35.283087 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:35.283286 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:35.283296 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:35.284395 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:35.284399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:35.285464 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:35.285478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:35.286256 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:35.286261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:35.286390 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:35.287221 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:35.287242 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:35.290684 (+ 3442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.121.190:45651: Server oryx/127.20.121.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":378,"threads_started":1}
W20260929 16:05:35.291275 30321 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:35.281184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:35181 (local address 127.20.121.190:45651)
0929 16:05:35.281420 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:35.281425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:35.282142 (+ 717us) server_negotiation.cc:407] Connection header received
0929 16:05:35.282312 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:35.282316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:35.282388 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:35.282497 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:35.283453 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:35.284225 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:35.285617 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:35.286103 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:35.286202 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:35.291006 (+ 4804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.121.130:35181: BlockingRecv error: recv got EOF from 127.20.121.130:35181 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 16:05:35.602636 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 29835
I20260929 16:05:35.620040 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:35.604195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.20.121.129:46833)
0929 16:05:35.604964 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:35.604984 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:35.605023 (+ 39us) server_negotiation.cc:407] Connection header received
0929 16:05:35.605112 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:35.605120 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:35.605419 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:35.605641 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:35.607537 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:35.609089 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:35.610764 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:35.611497 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:35.611648 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:35.614769 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:35.614784 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:35.614807 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:35.614867 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:35.619210 (+ 4343us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.121.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 16:05:35.619632 (+ 422us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54694: Request ticket server kudu/127.20.121.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":421,"threads_started":1}
W20260929 16:05:35.620268 30324 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54694: Request ticket server kudu/127.20.121.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 16:05:35.620311 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 29835: Not authorized: Client connection negotiation failed: client connection to 127.20.121.129:46833: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.121.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 16:05:35.620639 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 29835
I20260929 16:05:35.638509 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 29987
I20260929 16:05:35.658416 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:35.640191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51914 (local address 127.20.121.130:41687)
0929 16:05:35.641003 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:35.641014 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:35.641056 (+ 42us) server_negotiation.cc:407] Connection header received
0929 16:05:35.641139 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:35.641147 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:35.641557 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:35.641780 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:35.643282 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:35.644558 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:35.645873 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:35.646775 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:35.646908 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:35.651293 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:35.651320 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:35.651351 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:35.651473 (+ 122us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:35.657457 (+ 5984us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.121.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 16:05:35.657989 (+ 532us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51914: Request ticket server kudu/127.20.121.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":357,"threads_started":1}
W20260929 16:05:35.658552 30325 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51914: Request ticket server kudu/127.20.121.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 16:05:35.659044 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 29987: Not authorized: Client connection negotiation failed: client connection to 127.20.121.130:41687: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.121.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 16:05:35.659284 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 29987
I20260929 16:05:35.688097 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 30118
I20260929 16:05:35.707806 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:35.689588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49462 (local address 127.20.121.131:43421)
0929 16:05:35.690610 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:35.690626 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:35.690676 (+ 50us) server_negotiation.cc:407] Connection header received
0929 16:05:35.690798 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:35.690812 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:35.691232 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:35.691545 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:35.692971 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:35.695300 (+ 2329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:35.696849 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:35.698275 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:35.698516 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:35.701515 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:35.701536 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:35.701567 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:35.701642 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:35.706705 (+ 5063us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.121.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 16:05:35.707303 (+ 598us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49462: Request ticket server kudu/127.20.121.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":472,"threads_started":1}
W20260929 16:05:35.707942 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 30118: Not authorized: Client connection negotiation failed: client connection to 127.20.121.131:43421: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.20.121.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 16:05:35.708194 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 30118
I20260929 16:05:35.736338 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 29766
I20260929 16:05:35.737610 29828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:35.855791 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 29766
2026-09-29T16:05:35Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21470 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T16:05:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:05:35Z Disabled control of system clock
I20260929 16:05:35.938848 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:45547
--webserver_interface=127.20.121.190
--webserver_port=0
--builtin_ntp_servers=127.20.121.148:36181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:45547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 16:05:36.285777 30331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:36.286176 30331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:36.286267 30331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:36.293602 30331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:05:36.293720 30331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:36.293758 30331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:36.293807 30331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:05:36.293853 30331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:05:36.303427 30331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:36181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:45547
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:45547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:36.304725 30331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:36.306602 30331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:36.319861 30337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:36.319971 30336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:36.320104 30331 server_base.cc:1034] running on GCE node
W20260929 16:05:36.320354 30339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:36.321228 30331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:36.322813 30331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:36.324060 30331 hybrid_clock.cc:648] HybridClock initialized: now 1790697936323994 us; error 59 us; skew 500 ppm
I20260929 16:05:36.324427 30331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:05:36.327057 30331 webserver.cc:460] Webserver started at http://127.20.121.190:46627/ using document root <none> and password file <none>
I20260929 16:05:36.327855 30331 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:36.328003 30331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:36.328367 30331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:36.331624 30331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "474fc0f00f5d41439b9854d2511f9dbc"
format_stamp: "Formatted at 2026-09-29 16:05:36 on dist-test-slave-x07d"
server_key: "a6bc6c7abb1a404099336421729aab8e"
server_key_iv: "7ad5c06ab399aa8ac1063d4f6e8ed05e"
server_key_version: "encryptionkey@0"
I20260929 16:05:36.332535 30331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "474fc0f00f5d41439b9854d2511f9dbc"
format_stamp: "Formatted at 2026-09-29 16:05:36 on dist-test-slave-x07d"
server_key: "a6bc6c7abb1a404099336421729aab8e"
server_key_iv: "7ad5c06ab399aa8ac1063d4f6e8ed05e"
server_key_version: "encryptionkey@0"
I20260929 16:05:36.338369 30331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260929 16:05:36.342444 30345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:36.343961 30331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:05:36.344218 30331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "474fc0f00f5d41439b9854d2511f9dbc"
format_stamp: "Formatted at 2026-09-29 16:05:36 on dist-test-slave-x07d"
server_key: "a6bc6c7abb1a404099336421729aab8e"
server_key_iv: "7ad5c06ab399aa8ac1063d4f6e8ed05e"
server_key_version: "encryptionkey@0"
I20260929 16:05:36.344478 30331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:36.364295 30331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:36.365239 30331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:36.365581 30331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:36.385704 30331 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:45547
I20260929 16:05:36.385725 30396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:45547 every 8 connection(s)
I20260929 16:05:36.387319 30331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:05:36.391454 30397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:36.392807 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 30331
I20260929 16:05:36.393049 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:05:36.393775 20966 external_mini_cluster.cc:1428] Setting key 8c96465091306a6ab3194e0b58b081a4
I20260929 16:05:36.405930 30397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc: Bootstrap starting.
I20260929 16:05:36.408829 30400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:36.395994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41244 (local address 127.20.121.190:45547)
0929 16:05:36.397236 (+ 1242us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:36.397247 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:36.397298 (+ 51us) server_negotiation.cc:407] Connection header received
0929 16:05:36.399140 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:36.399171 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:36.399744 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:36.400188 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:36.400822 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:36.402332 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:36.403838 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:36.404649 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:36.404853 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:36.406261 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:36.406273 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:36.406290 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:36.406927 (+ 637us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:36.407009 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:36.407031 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:36.407168 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:36.407454 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 16:05:36.407797 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":389,"threads_started":1}
I20260929 16:05:36.409710 30397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:36.411002 30397 log.cc:824] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:36.414083 30397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc: No bootstrap required, opened a new log
I20260929 16:05:36.422260 30397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "474fc0f00f5d41439b9854d2511f9dbc" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45547 } }
I20260929 16:05:36.422821 30397 raft_consensus.cc:374] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:36.422964 30397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 474fc0f00f5d41439b9854d2511f9dbc, State: Initialized, Role: FOLLOWER
I20260929 16:05:36.423744 30397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "474fc0f00f5d41439b9854d2511f9dbc" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45547 } }
I20260929 16:05:36.424001 30397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:05:36.424181 30397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:05:36.424383 30397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:36.427016 30397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "474fc0f00f5d41439b9854d2511f9dbc" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45547 } }
I20260929 16:05:36.427704 30397 leader_election.cc:302] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 474fc0f00f5d41439b9854d2511f9dbc; no voters:
I20260929 16:05:36.428332 30397 leader_election.cc:288] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:05:36.428512 30402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:36.429868 30402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [term 1 LEADER]: Becoming Leader. State: Replica: 474fc0f00f5d41439b9854d2511f9dbc, State: Running, Role: LEADER
I20260929 16:05:36.430711 30402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "474fc0f00f5d41439b9854d2511f9dbc" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45547 } }
I20260929 16:05:36.431634 30397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:05:36.434052 30404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "474fc0f00f5d41439b9854d2511f9dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "474fc0f00f5d41439b9854d2511f9dbc" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45547 } } }
I20260929 16:05:36.434430 30404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:36.434119 30403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 474fc0f00f5d41439b9854d2511f9dbc. Latest consensus state: current_term: 1 leader_uuid: "474fc0f00f5d41439b9854d2511f9dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "474fc0f00f5d41439b9854d2511f9dbc" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45547 } } }
I20260929 16:05:36.434571 30403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:36.436510 30408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:05:36.442036 30408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:05:36.452867 30408 catalog_manager.cc:1347] Generated new cluster ID: bb2504600d564917b75a64a4082e8df3
I20260929 16:05:36.453004 30408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:05:36.484243 30408 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:05:36.486232 30408 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:05:36.504961 30408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 474fc0f00f5d41439b9854d2511f9dbc: Generated new TSK 0
I20260929 16:05:36.506256 30408 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:05:36.518399 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45547
--builtin_ntp_servers=127.20.121.148:36181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 16:05:36.889480 30421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:36.889789 30421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:36.889951 30421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:36.897794 30421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:36.897926 30421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:36.898068 30421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:05:36.907581 30421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:36181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:36.909138 30421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:36.911172 30421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:36.926229 30429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:36.926306 30426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:36.926210 30427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:36.929049 30421 server_base.cc:1034] running on GCE node
I20260929 16:05:36.929796 30421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:36.931063 30421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:36.932303 30421 hybrid_clock.cc:648] HybridClock initialized: now 1790697936932237 us; error 60 us; skew 500 ppm
I20260929 16:05:36.932698 30421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:05:36.943189 30421 webserver.cc:460] Webserver started at http://127.20.121.129:35925/ using document root <none> and password file <none>
I20260929 16:05:36.944020 30421 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:36.944134 30421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:36.944464 30421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:36.947860 30421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "c065d3aa88e9446fa11c9396dd61d49f"
format_stamp: "Formatted at 2026-09-29 16:05:36 on dist-test-slave-x07d"
server_key: "fc207d968797d2132b689a9a154e2981"
server_key_iv: "1b48f459f58dd003a4d6ee42f0d2bf94"
server_key_version: "encryptionkey@0"
I20260929 16:05:36.948848 30421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "c065d3aa88e9446fa11c9396dd61d49f"
format_stamp: "Formatted at 2026-09-29 16:05:36 on dist-test-slave-x07d"
server_key: "fc207d968797d2132b689a9a154e2981"
server_key_iv: "1b48f459f58dd003a4d6ee42f0d2bf94"
server_key_version: "encryptionkey@0"
I20260929 16:05:36.954888 30421 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260929 16:05:36.958495 30435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:36.960017 30421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 16:05:36.960188 30421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "c065d3aa88e9446fa11c9396dd61d49f"
format_stamp: "Formatted at 2026-09-29 16:05:36 on dist-test-slave-x07d"
server_key: "fc207d968797d2132b689a9a154e2981"
server_key_iv: "1b48f459f58dd003a4d6ee42f0d2bf94"
server_key_version: "encryptionkey@0"
I20260929 16:05:36.960412 30421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:36.981643 30421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:36.982627 30421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:36.982960 30421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:36.983999 30421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:36.985594 30421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:36.985724 30421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:36.985869 30421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:36.985966 30421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:37.025106 30421 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:38159
I20260929 16:05:37.025205 30547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:38159 every 8 connection(s)
I20260929 16:05:37.026604 30421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:05:37.032536 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 30421
I20260929 16:05:37.032804 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:05:37.033668 20966 external_mini_cluster.cc:1428] Setting key d60a57bcadbdf8390142b0b03f6403ab
I20260929 16:05:37.036671 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45547
--builtin_ntp_servers=127.20.121.148:36181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 16:05:37.041676 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:37.029435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:35295 (local address 127.20.121.190:45547)
0929 16:05:37.029936 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:37.029943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:37.031271 (+ 1328us) server_negotiation.cc:407] Connection header received
0929 16:05:37.033247 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:37.033256 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:37.033363 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:37.033520 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:37.034774 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:37.035768 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:37.038046 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:37.038890 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:37.039058 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:37.040519 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:37.040530 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:37.040539 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:37.040567 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:37.040620 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:37.040639 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:37.040741 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:37.041071 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 16:05:37.041196 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":233,"threads_started":1}
I20260929 16:05:37.042618 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:37.029938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45547 (local address 127.20.121.129:35295)
0929 16:05:37.031054 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:37.031110 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:37.032879 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:37.033710 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:37.033726 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:37.033876 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:37.034612 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:37.034629 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:37.036053 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:37.036058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:37.037824 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:37.037839 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:37.039167 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:37.039172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:37.039389 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:37.040159 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:05:37.040161 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:37.040205 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:05:37.040296 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:37.040771 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:37.040780 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:37.040787 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 16:05:37.041027 (+ 240us) client_negotiation.cc:240] Negotiation successful
0929 16:05:37.041378 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":304,"threads_started":1}
I20260929 16:05:37.044445 30548 heartbeater.cc:347] Connected to a master server at 127.20.121.190:45547
I20260929 16:05:37.044852 30548 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:37.045879 30548 heartbeater.cc:511] Master 127.20.121.190:45547 requested a full tablet report, sending...
I20260929 16:05:37.048373 30362 ts_manager.cc:194] Registered new tserver with Master: c065d3aa88e9446fa11c9396dd61d49f (127.20.121.129:38159)
I20260929 16:05:37.050335 30362 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.129:35295
W20260929 16:05:37.393908 30552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:37.394287 30552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:37.394446 30552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:37.401810 30552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:37.401974 30552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:37.402159 30552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:05:37.411751 30552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:36181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:37.413288 30552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:37.415148 30552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:37.428987 30557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:37.428934 30560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:37.429085 30558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:37.430058 30552 server_base.cc:1034] running on GCE node
I20260929 16:05:37.430737 30552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:37.431818 30552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:37.433063 30552 hybrid_clock.cc:648] HybridClock initialized: now 1790697937433004 us; error 55 us; skew 500 ppm
I20260929 16:05:37.433403 30552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:05:37.435954 30552 webserver.cc:460] Webserver started at http://127.20.121.130:33113/ using document root <none> and password file <none>
I20260929 16:05:37.436790 30552 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:37.436934 30552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:37.437280 30552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:37.440582 30552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "4e6d6f53a6e14aada2603fd4b97a9474"
format_stamp: "Formatted at 2026-09-29 16:05:37 on dist-test-slave-x07d"
server_key: "acda4eb435e0037074fe6d6b53de6095"
server_key_iv: "e38b5ca56f410a46edf57b8d9ff1a9d8"
server_key_version: "encryptionkey@0"
I20260929 16:05:37.441488 30552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "4e6d6f53a6e14aada2603fd4b97a9474"
format_stamp: "Formatted at 2026-09-29 16:05:37 on dist-test-slave-x07d"
server_key: "acda4eb435e0037074fe6d6b53de6095"
server_key_iv: "e38b5ca56f410a46edf57b8d9ff1a9d8"
server_key_version: "encryptionkey@0"
I20260929 16:05:37.447319 30552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 16:05:37.451741 30566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:37.453478 30552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260929 16:05:37.453707 30552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "4e6d6f53a6e14aada2603fd4b97a9474"
format_stamp: "Formatted at 2026-09-29 16:05:37 on dist-test-slave-x07d"
server_key: "acda4eb435e0037074fe6d6b53de6095"
server_key_iv: "e38b5ca56f410a46edf57b8d9ff1a9d8"
server_key_version: "encryptionkey@0"
I20260929 16:05:37.454077 30552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:37.493319 30552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:37.494374 30552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:37.494740 30552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:37.495894 30552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:37.497798 30552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:37.497954 30552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:37.498119 30552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:37.498220 30552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:37.534631 30552 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:39321
I20260929 16:05:37.534740 30678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:39321 every 8 connection(s)
I20260929 16:05:37.536003 30552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:05:37.545744 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 30552
I20260929 16:05:37.545969 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:05:37.546434 20966 external_mini_cluster.cc:1428] Setting key 86f0649e1fca295a5ed4474179f44abf
I20260929 16:05:37.549750 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45547
--builtin_ntp_servers=127.20.121.148:36181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 16:05:37.550364 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:37.538980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:34859 (local address 127.20.121.190:45547)
0929 16:05:37.539164 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:37.539169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:37.540638 (+ 1469us) server_negotiation.cc:407] Connection header received
0929 16:05:37.542680 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:37.542686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:37.542779 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:37.542897 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:37.543983 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:37.544751 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:37.546653 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:37.547089 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:37.547240 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:37.549170 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:37.549176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:37.549180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:37.549197 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:37.549253 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:37.549390 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:37.549525 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:37.549822 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 16:05:37.549976 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 16:05:37.551924 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:37.539510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45547 (local address 127.20.121.130:34859)
0929 16:05:37.540472 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:37.540510 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:37.542403 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:37.543136 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:37.543145 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:37.543234 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:37.543802 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:37.543818 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:37.544927 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:37.544933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:37.546437 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:37.546455 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:37.547316 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:37.547322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:37.547659 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:37.548727 (+ 1068us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:05:37.548731 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:37.548788 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:05:37.548897 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:37.549533 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:37.549547 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:37.549560 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 16:05:37.549786 (+ 226us) client_negotiation.cc:240] Negotiation successful
0929 16:05:37.550293 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":282,"threads_started":1}
I20260929 16:05:37.553454 30679 heartbeater.cc:347] Connected to a master server at 127.20.121.190:45547
I20260929 16:05:37.553786 30679 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:37.554688 30679 heartbeater.cc:511] Master 127.20.121.190:45547 requested a full tablet report, sending...
I20260929 16:05:37.556279 30362 ts_manager.cc:194] Registered new tserver with Master: 4e6d6f53a6e14aada2603fd4b97a9474 (127.20.121.130:39321)
I20260929 16:05:37.557076 30362 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.130:34859
W20260929 16:05:37.897843 30682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:37.898248 30682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:37.898399 30682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:37.905894 30682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:37.906060 30682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:37.906232 30682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:05:37.915724 30682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:36181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:45547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:37.917104 30682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:37.918927 30682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:37.932468 30687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:37.932446 30688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:37.932674 30690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:37.933525 30682 server_base.cc:1034] running on GCE node
I20260929 16:05:37.934265 30682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:37.935279 30682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:37.936501 30682 hybrid_clock.cc:648] HybridClock initialized: now 1790697937936455 us; error 46 us; skew 500 ppm
I20260929 16:05:37.936867 30682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:05:37.940222 30682 webserver.cc:460] Webserver started at http://127.20.121.131:45003/ using document root <none> and password file <none>
I20260929 16:05:37.941027 30682 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:37.941131 30682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:37.941521 30682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:37.944892 30682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "9b43b23651bb417fb49a1b37474b9dce"
format_stamp: "Formatted at 2026-09-29 16:05:37 on dist-test-slave-x07d"
server_key: "9813562dbc4b7f1291dbff9d60bdb95f"
server_key_iv: "0a2cfe5ece7e8585183b7d1634244e60"
server_key_version: "encryptionkey@0"
I20260929 16:05:37.945765 30682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "9b43b23651bb417fb49a1b37474b9dce"
format_stamp: "Formatted at 2026-09-29 16:05:37 on dist-test-slave-x07d"
server_key: "9813562dbc4b7f1291dbff9d60bdb95f"
server_key_iv: "0a2cfe5ece7e8585183b7d1634244e60"
server_key_version: "encryptionkey@0"
I20260929 16:05:37.952364 30682 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 16:05:37.956763 30696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:37.958381 30682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 16:05:37.958573 30682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "9b43b23651bb417fb49a1b37474b9dce"
format_stamp: "Formatted at 2026-09-29 16:05:37 on dist-test-slave-x07d"
server_key: "9813562dbc4b7f1291dbff9d60bdb95f"
server_key_iv: "0a2cfe5ece7e8585183b7d1634244e60"
server_key_version: "encryptionkey@0"
I20260929 16:05:37.958807 30682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:37.976439 30682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:37.977306 30682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:37.977591 30682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:37.978543 30682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:37.980114 30682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:37.980201 30682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:37.980286 30682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:37.980341 30682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:38.015082 30682 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:44029
I20260929 16:05:38.015192 30808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:44029 every 8 connection(s)
I20260929 16:05:38.016508 30682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:05:38.025810 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 30682
I20260929 16:05:38.026151 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:05:38.026835 20966 external_mini_cluster.cc:1428] Setting key b2397c0796615538bbf1d5b74a979375
I20260929 16:05:38.032483 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:38.019402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:60031 (local address 127.20.121.190:45547)
0929 16:05:38.019637 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:38.019644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:38.021255 (+ 1611us) server_negotiation.cc:407] Connection header received
0929 16:05:38.023253 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:38.023258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:38.023337 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:38.023467 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:38.024761 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:38.025954 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:38.028765 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:38.029441 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:38.029577 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:38.031490 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:38.031495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:38.031499 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:38.031517 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:38.031550 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:38.031563 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:38.031645 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:38.032115 (+ 470us) server_negotiation.cc:299] Negotiation successful
0929 16:05:38.032214 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 16:05:38.033742 30811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:38.019998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:45547 (local address 127.20.121.131:60031)
0929 16:05:38.021057 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:38.021108 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:38.022921 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:38.023776 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:38.023786 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:38.023891 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:38.024560 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:38.024578 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:38.026233 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:38.026244 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:38.028586 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:38.028609 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:38.029716 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:38.029725 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:38.030144 (+ 419us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:38.031100 (+ 956us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:05:38.031103 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:38.031150 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:05:38.031244 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:38.031663 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:38.031672 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:38.031678 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 16:05:38.031985 (+ 307us) client_negotiation.cc:240] Negotiation successful
0929 16:05:38.032419 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":332,"threads_started":1}
I20260929 16:05:38.035369 30809 heartbeater.cc:347] Connected to a master server at 127.20.121.190:45547
I20260929 16:05:38.035713 30809 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:38.036590 30809 heartbeater.cc:511] Master 127.20.121.190:45547 requested a full tablet report, sending...
I20260929 16:05:38.038313 30362 ts_manager.cc:194] Registered new tserver with Master: 9b43b23651bb417fb49a1b37474b9dce (127.20.121.131:44029)
I20260929 16:05:38.039094 30362 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.131:60031
I20260929 16:05:38.043349 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 16:05:38.050709 30550 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:38.048761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41250 (local address 127.20.121.190:45547)
0929 16:05:38.049084 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:38.049094 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:38.049494 (+ 400us) server_negotiation.cc:407] Connection header received
0929 16:05:38.049721 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:38.049732 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:38.049803 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:38.049928 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:38.050497 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41250: BlockingRecv error: recv got EOF from 127.0.0.1:41250 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 16:05:38.051478 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 30421
I20260929 16:05:38.054759 30548 heartbeater.cc:503] Master 127.20.121.190:45547 was elected leader, sending a full tablet report...
I20260929 16:05:38.061234 30818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:38.053043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.20.121.129:38159)
0929 16:05:38.053685 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:38.053693 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:38.053722 (+ 29us) server_negotiation.cc:407] Connection header received
0929 16:05:38.053784 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:38.053791 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:38.054094 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:38.054245 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:38.054717 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:38.056033 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:38.057598 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:38.058413 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:38.058523 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:38.059700 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:38.059709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:38.059735 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:38.060430 (+ 695us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:38.060489 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:38.060511 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:38.060604 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:38.060761 (+ 157us) server_negotiation.cc:299] Negotiation successful
0929 16:05:38.060884 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":319,"threads_started":1}
I20260929 16:05:38.062597 30543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:38.150647 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 30421
I20260929 16:05:38.168638 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 30552
I20260929 16:05:38.178618 30820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:38.169674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57988 (local address 127.20.121.130:39321)
0929 16:05:38.170257 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:38.170265 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:38.170296 (+ 31us) server_negotiation.cc:407] Connection header received
0929 16:05:38.170368 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:38.170374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:38.170602 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:38.170746 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:38.171456 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:38.172972 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:38.174602 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:38.175343 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:38.175514 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:38.176762 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:38.176772 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:38.176794 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:38.177580 (+ 786us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:38.177633 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:38.177655 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:38.177816 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:38.177991 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 16:05:38.178178 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":251,"threads_started":1}
I20260929 16:05:38.180090 30674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:38.264114 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 30552
I20260929 16:05:38.280421 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 30682
I20260929 16:05:38.289332 30822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:38.281454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59386 (local address 127.20.121.131:44029)
0929 16:05:38.282157 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:38.282165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:38.282197 (+ 32us) server_negotiation.cc:407] Connection header received
0929 16:05:38.282270 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:38.282277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:38.282513 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:38.282693 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:38.283165 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:38.284293 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:38.285650 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:38.286406 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:38.286556 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:38.287722 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:38.287732 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:38.287747 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:38.288381 (+ 634us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:38.288464 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:38.288494 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:38.288640 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:38.288790 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 16:05:38.288946 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":312,"threads_started":1}
I20260929 16:05:38.290648 30804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:38.375448 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 30682
I20260929 16:05:38.392447 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 30331
I20260929 16:05:38.393668 30392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:38.498128 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 30331
2026-09-29T16:05:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2634 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T16:05:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:05:38Z Disabled control of system clock
I20260929 16:05:38.573800 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37363
--webserver_interface=127.20.121.190
--webserver_port=0
--builtin_ntp_servers=127.20.121.148:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:37363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 16:05:38.917944 30828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:38.918284 30828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:38.918430 30828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:38.925943 30828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:05:38.926097 30828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:38.926136 30828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:38.926254 30828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:05:38.926328 30828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:05:38.935817 30828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:37363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:38.937216 30828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:38.939172 30828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:38.951408 30834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:38.951496 30833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:38.951472 30836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:38.952118 30828 server_base.cc:1034] running on GCE node
I20260929 16:05:38.953088 30828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:38.954821 30828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:38.956081 30828 hybrid_clock.cc:648] HybridClock initialized: now 1790697938956014 us; error 54 us; skew 500 ppm
I20260929 16:05:38.956481 30828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:05:38.958940 30828 webserver.cc:460] Webserver started at http://127.20.121.190:37623/ using document root <none> and password file <none>
I20260929 16:05:38.959717 30828 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:38.959842 30828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:38.960162 30828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:38.963184 30828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "c2d24cf235c943e8a2ef078786f04960"
format_stamp: "Formatted at 2026-09-29 16:05:38 on dist-test-slave-x07d"
server_key: "56acba40f44b518b4c5f10d1b5406498"
server_key_iv: "03f9c7caa0c9936d6e115a8e85d37222"
server_key_version: "encryptionkey@0"
I20260929 16:05:38.964003 30828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "c2d24cf235c943e8a2ef078786f04960"
format_stamp: "Formatted at 2026-09-29 16:05:38 on dist-test-slave-x07d"
server_key: "56acba40f44b518b4c5f10d1b5406498"
server_key_iv: "03f9c7caa0c9936d6e115a8e85d37222"
server_key_version: "encryptionkey@0"
I20260929 16:05:38.969167 30828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 16:05:38.972916 30842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:38.974476 30828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:05:38.974673 30828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "c2d24cf235c943e8a2ef078786f04960"
format_stamp: "Formatted at 2026-09-29 16:05:38 on dist-test-slave-x07d"
server_key: "56acba40f44b518b4c5f10d1b5406498"
server_key_iv: "03f9c7caa0c9936d6e115a8e85d37222"
server_key_version: "encryptionkey@0"
I20260929 16:05:38.974902 30828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:38.980822 30828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:38.981628 30828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:38.981932 30828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:39.000650 30828 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:37363
I20260929 16:05:39.000684 30893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:37363 every 8 connection(s)
I20260929 16:05:39.002625 30828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:05:39.006935 30894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:39.010649 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 30828
I20260929 16:05:39.010854 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:05:39.011739 20966 external_mini_cluster.cc:1428] Setting key 7c86906ade617ba166753afb9f6a4eb2
I20260929 16:05:39.021607 30894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960: Bootstrap starting.
I20260929 16:05:39.022254 30897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:39.014334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.20.121.190:37363)
0929 16:05:39.015590 (+ 1256us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:39.015602 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:39.015648 (+ 46us) server_negotiation.cc:407] Connection header received
0929 16:05:39.017401 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:39.017436 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:39.017963 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:39.018881 (+ 918us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:39.020179 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:39.020186 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:39.020200 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:39.020219 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:39.020276 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:39.020293 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:39.020564 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:39.020712 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 16:05:39.021121 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":421,"threads_started":1}
I20260929 16:05:39.024979 30894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:39.026265 30894 log.cc:824] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:39.029668 30894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960: No bootstrap required, opened a new log
I20260929 16:05:39.037851 30894 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d24cf235c943e8a2ef078786f04960" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37363 } }
I20260929 16:05:39.038386 30894 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:39.038492 30894 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d24cf235c943e8a2ef078786f04960, State: Initialized, Role: FOLLOWER
I20260929 16:05:39.039112 30894 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d24cf235c943e8a2ef078786f04960" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37363 } }
I20260929 16:05:39.039366 30894 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:05:39.039511 30894 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:05:39.039705 30894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:39.042411 30894 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d24cf235c943e8a2ef078786f04960" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37363 } }
I20260929 16:05:39.043030 30894 leader_election.cc:302] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2d24cf235c943e8a2ef078786f04960; no voters:
I20260929 16:05:39.043671 30894 leader_election.cc:288] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:05:39.043939 30899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:39.044747 30899 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [term 1 LEADER]: Becoming Leader. State: Replica: c2d24cf235c943e8a2ef078786f04960, State: Running, Role: LEADER
I20260929 16:05:39.045329 30899 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d24cf235c943e8a2ef078786f04960" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37363 } }
I20260929 16:05:39.045938 30894 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:05:39.048550 30901 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2d24cf235c943e8a2ef078786f04960. Latest consensus state: current_term: 1 leader_uuid: "c2d24cf235c943e8a2ef078786f04960" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d24cf235c943e8a2ef078786f04960" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37363 } } }
I20260929 16:05:39.048600 30900 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2d24cf235c943e8a2ef078786f04960" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2d24cf235c943e8a2ef078786f04960" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37363 } } }
I20260929 16:05:39.048983 30901 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:39.049044 30900 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960 [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:39.051209 30907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:05:39.056988 30907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:05:39.067147 30907 catalog_manager.cc:1347] Generated new cluster ID: d7f0de62d3ce4d8f84d643a961db07f3
I20260929 16:05:39.067288 30907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:05:39.093729 30907 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:05:39.094425 30907 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:05:39.112385 30907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2d24cf235c943e8a2ef078786f04960: Generated new TSK 0
I20260929 16:05:39.113686 30907 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:05:39.120502 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37363
--builtin_ntp_servers=127.20.121.148:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 16:05:39.475104 30918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:39.475476 30918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:39.475660 30918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:39.483654 30918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:39.483832 30918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:39.484011 30918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:05:39.493824 30918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:39.495302 30918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:39.497360 30918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:39.512456 30923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:39.512578 30926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:39.512697 30924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:39.514519 30918 server_base.cc:1034] running on GCE node
I20260929 16:05:39.515277 30918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:39.516438 30918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:39.517717 30918 hybrid_clock.cc:648] HybridClock initialized: now 1790697939517668 us; error 52 us; skew 500 ppm
I20260929 16:05:39.518213 30918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:05:39.521035 30918 webserver.cc:460] Webserver started at http://127.20.121.129:38725/ using document root <none> and password file <none>
I20260929 16:05:39.521862 30918 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:39.521962 30918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:39.522367 30918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:39.525574 30918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "cb7f5c5ad30349e6aaf65fbeb7935d1f"
format_stamp: "Formatted at 2026-09-29 16:05:39 on dist-test-slave-x07d"
server_key: "54ded92bec5d487c9eeab9419c176ecd"
server_key_iv: "ad16e221b6a7708dd1d5f9ee4b9b8797"
server_key_version: "encryptionkey@0"
I20260929 16:05:39.526522 30918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "cb7f5c5ad30349e6aaf65fbeb7935d1f"
format_stamp: "Formatted at 2026-09-29 16:05:39 on dist-test-slave-x07d"
server_key: "54ded92bec5d487c9eeab9419c176ecd"
server_key_iv: "ad16e221b6a7708dd1d5f9ee4b9b8797"
server_key_version: "encryptionkey@0"
I20260929 16:05:39.532205 30918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260929 16:05:39.536182 30932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:39.537710 30918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:05:39.537899 30918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "cb7f5c5ad30349e6aaf65fbeb7935d1f"
format_stamp: "Formatted at 2026-09-29 16:05:39 on dist-test-slave-x07d"
server_key: "54ded92bec5d487c9eeab9419c176ecd"
server_key_iv: "ad16e221b6a7708dd1d5f9ee4b9b8797"
server_key_version: "encryptionkey@0"
I20260929 16:05:39.538251 30918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:39.544513 30918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:39.545433 30918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:39.545742 30918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:39.546953 30918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:39.548599 30918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:39.548718 30918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:39.548857 30918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:39.548942 30918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:39.584779 30918 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:45495
I20260929 16:05:39.584867 31044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:45495 every 8 connection(s)
I20260929 16:05:39.586228 30918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:05:39.595146 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:39.588974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:47969 (local address 127.20.121.190:37363)
0929 16:05:39.589477 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:39.589483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:39.590992 (+ 1509us) server_negotiation.cc:407] Connection header received
0929 16:05:39.592748 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:39.592754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:39.592826 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:39.592965 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:39.594356 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:39.594361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:39.594364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:39.594381 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:39.594420 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:39.594432 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:39.594578 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:39.594763 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 16:05:39.594832 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":256,"threads_started":1}
I20260929 16:05:39.595744 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 30918
I20260929 16:05:39.595943 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:05:39.596199 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:39.589674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37363 (local address 127.20.121.129:47969)
0929 16:05:39.590785 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:39.590834 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:39.592440 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:39.593207 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:39.593216 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:39.593323 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:39.594048 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:05:39.594055 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:39.594096 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:05:39.594183 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:39.594546 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:39.594556 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:39.594564 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 16:05:39.594851 (+ 287us) client_negotiation.cc:240] Negotiation successful
0929 16:05:39.595130 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":384,"threads_started":1}
I20260929 16:05:39.596752 20966 external_mini_cluster.cc:1428] Setting key 7ef4f301c6776256b4c0936bb63d44e7
I20260929 16:05:39.598260 31045 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37363
I20260929 16:05:39.598670 31045 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:39.599601 31045 heartbeater.cc:511] Master 127.20.121.190:37363 requested a full tablet report, sending...
I20260929 16:05:39.600226 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37363
--builtin_ntp_servers=127.20.121.148:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 16:05:39.602178 30859 ts_manager.cc:194] Registered new tserver with Master: cb7f5c5ad30349e6aaf65fbeb7935d1f (127.20.121.129:45495)
W20260929 16:05:39.941862 31049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:39.942286 31049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:39.942452 31049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:39.950474 31049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:39.950587 31049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:39.950786 31049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:05:39.960611 31049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:39.961992 31049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:39.963840 31049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:39.977358 31054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:39.977329 31055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:39.977396 31057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:39.978247 31049 server_base.cc:1034] running on GCE node
I20260929 16:05:39.978827 31049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:39.979842 31049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:39.981058 31049 hybrid_clock.cc:648] HybridClock initialized: now 1790697939981006 us; error 44 us; skew 500 ppm
I20260929 16:05:39.981415 31049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:05:39.983937 31049 webserver.cc:460] Webserver started at http://127.20.121.130:37355/ using document root <none> and password file <none>
I20260929 16:05:39.984681 31049 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:39.984771 31049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:39.985118 31049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:39.988139 31049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "043bcd05713e40678ef619b164a68a15"
format_stamp: "Formatted at 2026-09-29 16:05:39 on dist-test-slave-x07d"
server_key: "3e37e80166c2f51458b7e559975f366f"
server_key_iv: "e3d4d0f759ee997d991d3deb33a55b62"
server_key_version: "encryptionkey@0"
I20260929 16:05:39.988996 31049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "043bcd05713e40678ef619b164a68a15"
format_stamp: "Formatted at 2026-09-29 16:05:39 on dist-test-slave-x07d"
server_key: "3e37e80166c2f51458b7e559975f366f"
server_key_iv: "e3d4d0f759ee997d991d3deb33a55b62"
server_key_version: "encryptionkey@0"
I20260929 16:05:39.994645 31049 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 16:05:39.998544 31063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:40.000028 31049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 16:05:40.000231 31049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "043bcd05713e40678ef619b164a68a15"
format_stamp: "Formatted at 2026-09-29 16:05:39 on dist-test-slave-x07d"
server_key: "3e37e80166c2f51458b7e559975f366f"
server_key_iv: "e3d4d0f759ee997d991d3deb33a55b62"
server_key_version: "encryptionkey@0"
I20260929 16:05:40.000476 31049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:40.006675 31049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:40.007552 31049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:40.007833 31049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:40.008898 31049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:40.010496 31049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:40.010596 31049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:40.010747 31049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:40.010831 31049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:40.044803 31049 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:37289
I20260929 16:05:40.044853 31175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:37289 every 8 connection(s)
I20260929 16:05:40.046332 31049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:05:40.054747 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:40.049008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:45177 (local address 127.20.121.190:37363)
0929 16:05:40.049224 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:40.049231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:40.050827 (+ 1596us) server_negotiation.cc:407] Connection header received
0929 16:05:40.052596 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:40.052601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:40.052668 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:40.052758 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:40.054118 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:40.054123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:40.054126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:40.054141 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:40.054170 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:40.054182 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:40.054282 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:40.054457 (+ 175us) server_negotiation.cc:299] Negotiation successful
0929 16:05:40.054516 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 16:05:40.055600 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 31049
I20260929 16:05:40.055768 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:05:40.056264 31178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:40.049603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37363 (local address 127.20.121.130:45177)
0929 16:05:40.050671 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:40.050709 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:40.052350 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:40.052971 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:40.052979 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:40.053080 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:40.053744 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:05:40.053747 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:40.053784 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:05:40.053870 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:40.054274 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:40.054282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:40.054289 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 16:05:40.054528 (+ 239us) client_negotiation.cc:240] Negotiation successful
0929 16:05:40.054741 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":327,"threads_started":1}
I20260929 16:05:40.056423 20966 external_mini_cluster.cc:1428] Setting key 141dc22b4ce8df3e729dcf73bd751c45
I20260929 16:05:40.058257 31176 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37363
I20260929 16:05:40.058564 31176 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:40.059175 31176 heartbeater.cc:511] Master 127.20.121.190:37363 requested a full tablet report, sending...
I20260929 16:05:40.059443 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37363
--builtin_ntp_servers=127.20.121.148:44315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 16:05:40.061002 30859 ts_manager.cc:194] Registered new tserver with Master: 043bcd05713e40678ef619b164a68a15 (127.20.121.130:37289)
W20260929 16:05:40.409677 31179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:40.410001 31179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:40.410195 31179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:40.417768 31179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:40.417904 31179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:40.418111 31179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:05:40.427752 31179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:44315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:40.429119 31179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:40.430956 31179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:40.445410 31184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:40.445479 31187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:40.445505 31185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:40.446702 31179 server_base.cc:1034] running on GCE node
I20260929 16:05:40.447352 31179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:40.448390 31179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:40.449591 31179 hybrid_clock.cc:648] HybridClock initialized: now 1790697940449541 us; error 49 us; skew 500 ppm
I20260929 16:05:40.449939 31179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:05:40.452540 31179 webserver.cc:460] Webserver started at http://127.20.121.131:46571/ using document root <none> and password file <none>
I20260929 16:05:40.453271 31179 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:40.453388 31179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:40.453719 31179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:40.456787 31179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "5787b2c7cbc549a0abb328465caed1d7"
format_stamp: "Formatted at 2026-09-29 16:05:40 on dist-test-slave-x07d"
server_key: "f89c78c4d3e9ebcab394a695be3b1f1e"
server_key_iv: "ec46ca229d7cd626d3d59786509b958b"
server_key_version: "encryptionkey@0"
I20260929 16:05:40.457684 31179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "5787b2c7cbc549a0abb328465caed1d7"
format_stamp: "Formatted at 2026-09-29 16:05:40 on dist-test-slave-x07d"
server_key: "f89c78c4d3e9ebcab394a695be3b1f1e"
server_key_iv: "ec46ca229d7cd626d3d59786509b958b"
server_key_version: "encryptionkey@0"
I20260929 16:05:40.463387 31179 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 16:05:40.467124 31193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:40.468679 31179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:05:40.468961 31179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "5787b2c7cbc549a0abb328465caed1d7"
format_stamp: "Formatted at 2026-09-29 16:05:40 on dist-test-slave-x07d"
server_key: "f89c78c4d3e9ebcab394a695be3b1f1e"
server_key_iv: "ec46ca229d7cd626d3d59786509b958b"
server_key_version: "encryptionkey@0"
I20260929 16:05:40.469205 31179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:40.475246 31179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:40.476188 31179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:40.476461 31179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:40.477407 31179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:40.479039 31179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:40.479127 31179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:40.479249 31179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:40.479322 31179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:40.513748 31179 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:37419
I20260929 16:05:40.513799 31305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:37419 every 8 connection(s)
I20260929 16:05:40.515116 31179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:05:40.523386 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 31179
I20260929 16:05:40.523569 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:05:40.523732 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:40.517828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:38465 (local address 127.20.121.190:37363)
0929 16:05:40.517992 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:40.517997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:40.519536 (+ 1539us) server_negotiation.cc:407] Connection header received
0929 16:05:40.521273 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:40.521279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:40.521357 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:40.521464 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:40.522889 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:40.522895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:40.522900 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:40.522927 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:40.522979 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:40.522997 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:40.523143 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:40.523353 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 16:05:40.523424 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 16:05:40.524120 20966 external_mini_cluster.cc:1428] Setting key d2b652eef9c3c1e099be8cbf94113534
I20260929 16:05:40.524801 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:40.518442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37363 (local address 127.20.121.131:38465)
0929 16:05:40.519369 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:40.519406 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:40.520977 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:40.521714 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:40.521722 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:40.521820 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:40.522583 (+ 763us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:05:40.522585 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:40.522626 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:05:40.522713 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:40.523129 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:40.523138 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:40.523145 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 16:05:40.523389 (+ 244us) client_negotiation.cc:240] Negotiation successful
0929 16:05:40.523709 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":262,"threads_started":1}
I20260929 16:05:40.526338 31306 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37363
I20260929 16:05:40.526664 31306 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:40.527294 31306 heartbeater.cc:511] Master 127.20.121.190:37363 requested a full tablet report, sending...
I20260929 16:05:40.528684 30859 ts_manager.cc:194] Registered new tserver with Master: 5787b2c7cbc549a0abb328465caed1d7 (127.20.121.131:37419)
I20260929 16:05:40.539513 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 16:05:40.545724 31047 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:05:40.544253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.20.121.190:37363)
0929 16:05:40.544409 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:40.544414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:40.544980 (+ 566us) server_negotiation.cc:407] Connection header received
0929 16:05:40.545125 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:40.545130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:40.545183 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:40.545276 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:40.545534 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60722: BlockingRecv error: recv got EOF from 127.0.0.1:60722 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 16:05:40.546743 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 30918
I20260929 16:05:40.549979 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:40.547534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50856 (local address 127.20.121.129:45495)
0929 16:05:40.548040 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:40.548055 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:40.548082 (+ 27us) server_negotiation.cc:407] Connection header received
0929 16:05:40.548139 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:40.548145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:40.548352 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:40.548486 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:40.549345 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:40.549352 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:40.549363 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:40.549380 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:40.549417 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:40.549430 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:40.549564 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:40.549646 (+ 82us) server_negotiation.cc:299] Negotiation successful
0929 16:05:40.549725 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":239,"threads_started":1}
I20260929 16:05:40.551223 31040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:40.631209 31045 heartbeater.cc:503] Master 127.20.121.190:37363 was elected leader, sending a full tablet report...
I20260929 16:05:40.633508 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 30918
I20260929 16:05:40.650560 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 31049
I20260929 16:05:40.654371 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:40.651624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.20.121.130:37289)
0929 16:05:40.652205 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:40.652220 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:40.652248 (+ 28us) server_negotiation.cc:407] Connection header received
0929 16:05:40.652310 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:40.652316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:40.652535 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:40.652696 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:40.653620 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:40.653626 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:40.653638 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:40.653654 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:40.653693 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:40.653706 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:40.653811 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:40.653906 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 16:05:40.653987 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":305,"threads_started":1}
I20260929 16:05:40.655678 31171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:40.738175 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 31049
I20260929 16:05:40.754876 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 31179
I20260929 16:05:40.758790 31319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:40.755812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42208 (local address 127.20.121.131:37419)
0929 16:05:40.756433 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:40.756441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:40.756471 (+ 30us) server_negotiation.cc:407] Connection header received
0929 16:05:40.756538 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:40.756544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:40.756769 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:40.756966 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:40.758062 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:40.758069 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:40.758081 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:05:40.758099 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:40.758140 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:05:40.758156 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:40.758284 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:40.758339 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 16:05:40.758440 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":314,"threads_started":1}
I20260929 16:05:40.760120 31301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:40.841506 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 31179
I20260929 16:05:40.857268 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 30828
I20260929 16:05:40.858565 30889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:40.959110 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 30828
2026-09-29T16:05:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2461 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:05:41 dist-test-slave-x07d krb5kdc[31324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:05:41 dist-test-slave-x07d krb5kdc[31324](info): set up 2 sockets
Sep 29 16:05:41 dist-test-slave-x07d krb5kdc[31324](info): commencing operation
krb5kdc: starting...
W20260929 16:05:42.075062 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:05:42 dist-test-slave-x07d krb5kdc[31324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:05:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:05:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:05:42.326223 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37431
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:37431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:42.663525 31337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:42.663851 31337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:42.664026 31337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:42.671618 31337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:05:42.671746 31337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:42.671792 31337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:42.671914 31337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:05:42.671993 31337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:05:42.682217 31337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:37431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:42.683683 31337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:42.685024 31337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:42.697757 31342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:42.697860 31343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:42.697860 31345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:42.698722 31337 server_base.cc:1034] running on GCE node
I20260929 16:05:42.699622 31337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:42.701267 31337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:42.702560 31337 hybrid_clock.cc:648] HybridClock initialized: now 1790697942702479 us; error 59 us; skew 500 ppm
I20260929 16:05:42.702953 31337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:42 dist-test-slave-x07d krb5kdc[31324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697942, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:42.713421 31337 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:05:42.715301 31337 webserver.cc:460] Webserver started at http://127.20.121.190:37025/ using document root <none> and password file <none>
I20260929 16:05:42.716074 31337 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:42.716239 31337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:42.716572 31337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:42.719784 31337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "f0cbd466ebdf4a60b716a08a0095a365"
format_stamp: "Formatted at 2026-09-29 16:05:42 on dist-test-slave-x07d"
server_key: "42ba5037b1b7b9fc520dbd2cd7c41f31"
server_key_iv: "e3824b6ed6884536c2d4fa3a8686081d"
server_key_version: "encryptionkey@0"
I20260929 16:05:42.720602 31337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "f0cbd466ebdf4a60b716a08a0095a365"
format_stamp: "Formatted at 2026-09-29 16:05:42 on dist-test-slave-x07d"
server_key: "42ba5037b1b7b9fc520dbd2cd7c41f31"
server_key_iv: "e3824b6ed6884536c2d4fa3a8686081d"
server_key_version: "encryptionkey@0"
I20260929 16:05:42.727031 31337 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260929 16:05:42.730738 31352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:42.732223 31337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:05:42.732426 31337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "f0cbd466ebdf4a60b716a08a0095a365"
format_stamp: "Formatted at 2026-09-29 16:05:42 on dist-test-slave-x07d"
server_key: "42ba5037b1b7b9fc520dbd2cd7c41f31"
server_key_iv: "e3824b6ed6884536c2d4fa3a8686081d"
server_key_version: "encryptionkey@0"
I20260929 16:05:42.732677 31337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:42.754086 31337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:42.758481 31337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:42.758792 31337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:42.778075 31337 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:37431
I20260929 16:05:42.778162 31403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:37431 every 8 connection(s)
I20260929 16:05:42.779448 31337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:05:42.780158 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 31337
I20260929 16:05:42.780392 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:05:42.781297 20966 external_mini_cluster.cc:1428] Setting key 68907a1d9b9d93d678279706fdee351b
I20260929 16:05:42.784178 31404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:42.797070 31404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365: Bootstrap starting.
Sep 29 16:05:42 dist-test-slave-x07d krb5kdc[31324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:05:42.801267 31404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:42.802740 31404 log.cc:824] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:42.806484 31404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365: No bootstrap required, opened a new log
I20260929 16:05:42.807564 31407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:42.783612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.20.121.190:37431)
0929 16:05:42.784402 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:42.784417 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:42.784475 (+ 58us) server_negotiation.cc:407] Connection header received
0929 16:05:42.786744 (+ 2269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:42.786771 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:42.787299 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:42.787744 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:42.789201 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:42.791204 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:42.792638 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:42.793682 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:42.793877 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:42.798429 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:42.798439 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:42.798459 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:42.798517 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:42.802706 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:42.803636 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:42.803643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:42.803649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:42.803814 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:42.804263 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:42.804267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:42.804270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:42.804950 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:42.805272 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:42.805663 (+ 391us) server_negotiation.cc:299] Negotiation successful
0929 16:05:42.806181 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":331,"threads_started":1}
I20260929 16:05:42.815887 31404 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cbd466ebdf4a60b716a08a0095a365" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37431 } }
I20260929 16:05:42.816442 31404 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:42.816587 31404 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0cbd466ebdf4a60b716a08a0095a365, State: Initialized, Role: FOLLOWER
I20260929 16:05:42.817246 31404 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cbd466ebdf4a60b716a08a0095a365" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37431 } }
I20260929 16:05:42.817497 31404 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:05:42.817647 31404 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:05:42.817833 31404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:42.820202 31404 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cbd466ebdf4a60b716a08a0095a365" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37431 } }
I20260929 16:05:42.820767 31404 leader_election.cc:302] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0cbd466ebdf4a60b716a08a0095a365; no voters:
I20260929 16:05:42.821448 31404 leader_election.cc:288] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:05:42.821622 31409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:42.822324 31409 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [term 1 LEADER]: Becoming Leader. State: Replica: f0cbd466ebdf4a60b716a08a0095a365, State: Running, Role: LEADER
I20260929 16:05:42.823098 31409 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cbd466ebdf4a60b716a08a0095a365" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37431 } }
I20260929 16:05:42.824266 31404 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:05:42.826046 31411 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0cbd466ebdf4a60b716a08a0095a365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cbd466ebdf4a60b716a08a0095a365" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37431 } } }
I20260929 16:05:42.826423 31411 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:42.826254 31410 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0cbd466ebdf4a60b716a08a0095a365. Latest consensus state: current_term: 1 leader_uuid: "f0cbd466ebdf4a60b716a08a0095a365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cbd466ebdf4a60b716a08a0095a365" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37431 } } }
I20260929 16:05:42.826520 31410 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365 [sys.catalog]: This master's current role is: LEADER
I20260929 16:05:42.830248 31415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:05:42.837755 31415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:05:42.847283 31415 catalog_manager.cc:1347] Generated new cluster ID: af161204f3da41f8903e1c561d638ccf
I20260929 16:05:42.847419 31415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:05:42.877678 31415 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:05:42.879669 31415 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:05:42.892107 31415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365: Generated new TSK 0
I20260929 16:05:42.893283 31415 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:05:42.967660 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37431
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:43.319562 31430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:43.319850 31430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:43.320026 31430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:43.327615 31430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:43.327731 31430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:43.327922 31430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:05:43.338851 31430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:43.340166 31430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:43.341452 31430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:43.355031 31436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:43.355208 31438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:43.355029 31435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:43.355903 31430 server_base.cc:1034] running on GCE node
I20260929 16:05:43.356571 31430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:43.357618 31430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:43.358837 31430 hybrid_clock.cc:648] HybridClock initialized: now 1790697943358792 us; error 38 us; skew 500 ppm
I20260929 16:05:43.359177 31430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:43 dist-test-slave-x07d krb5kdc[31324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:43.362432 31430 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:05:43.363958 31430 webserver.cc:460] Webserver started at http://127.20.121.129:40783/ using document root <none> and password file <none>
I20260929 16:05:43.364662 31430 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:43.364797 31430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:43.365089 31430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:43.368049 31430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "212719921ce346fcbeb7a8a890be2e0e"
format_stamp: "Formatted at 2026-09-29 16:05:43 on dist-test-slave-x07d"
server_key: "df622a57fc986d7a2efc73204d2d6050"
server_key_iv: "0c88a5bc02a759ef69e809e6ad426c52"
server_key_version: "encryptionkey@0"
I20260929 16:05:43.368844 31430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "212719921ce346fcbeb7a8a890be2e0e"
format_stamp: "Formatted at 2026-09-29 16:05:43 on dist-test-slave-x07d"
server_key: "df622a57fc986d7a2efc73204d2d6050"
server_key_iv: "0c88a5bc02a759ef69e809e6ad426c52"
server_key_version: "encryptionkey@0"
I20260929 16:05:43.374066 31430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 16:05:43.377439 31445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:43.378867 31430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:05:43.379037 31430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "212719921ce346fcbeb7a8a890be2e0e"
format_stamp: "Formatted at 2026-09-29 16:05:43 on dist-test-slave-x07d"
server_key: "df622a57fc986d7a2efc73204d2d6050"
server_key_iv: "0c88a5bc02a759ef69e809e6ad426c52"
server_key_version: "encryptionkey@0"
I20260929 16:05:43.379232 31430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:43.402665 31430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:43.407298 31430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:43.407620 31430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:43.408625 31430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:43.410091 31430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:43.410190 31430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:43.410336 31430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:43.410410 31430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:43.445413 31430 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:33315
I20260929 16:05:43.445465 31557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:33315 every 8 connection(s)
I20260929 16:05:43.446817 31430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:05:43.454929 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 31430
I20260929 16:05:43.455154 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:05:43.455771 20966 external_mini_cluster.cc:1428] Setting key f548007dd6b2475004d6590a67074a7a
Sep 29 16:05:43 dist-test-slave-x07d krb5kdc[31324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:05:43.469662 31560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:43.449474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:53243 (local address 127.20.121.190:37431)
0929 16:05:43.450035 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:43.450043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:43.451020 (+ 977us) server_negotiation.cc:407] Connection header received
0929 16:05:43.452882 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:43.452895 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:43.452964 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:43.453066 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:43.454950 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:43.455755 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:43.457897 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:43.458392 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:43.458507 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:43.463104 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:43.463109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:43.463113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:43.463147 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:43.466324 (+ 3177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:43.467193 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:43.467199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:43.467203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:43.467333 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:43.467807 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:43.467812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:43.467820 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:43.468100 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:43.468257 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:43.469043 (+ 786us) server_negotiation.cc:299] Negotiation successful
0929 16:05:43.469297 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":233,"threads_started":1}
I20260929 16:05:43.470788 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:43.450099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37431 (local address 127.20.121.129:53243)
0929 16:05:43.450844 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:43.450888 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:43.452582 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:43.453285 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:43.453294 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:43.453842 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:43.454659 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:43.454692 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:43.455972 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:43.455977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:43.457752 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:43.457765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:43.458688 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:43.458697 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:43.459053 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:43.459901 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:43.459940 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:43.462833 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:43.466501 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:43.466512 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:43.466568 (+ 56us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:43.467039 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:43.467457 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:43.467462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:43.467466 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:43.467679 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:43.468235 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:43.468244 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:43.468631 (+ 387us) client_negotiation.cc:769] Sending connection context
0929 16:05:43.468916 (+ 285us) client_negotiation.cc:240] Negotiation successful
0929 16:05:43.469499 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":275,"threads_started":1}
I20260929 16:05:43.472519 31558 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37431
I20260929 16:05:43.472867 31558 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:43.473814 31558 heartbeater.cc:511] Master 127.20.121.190:37431 requested a full tablet report, sending...
I20260929 16:05:43.476504 31369 ts_manager.cc:194] Registered new tserver with Master: 212719921ce346fcbeb7a8a890be2e0e (127.20.121.129:33315)
I20260929 16:05:43.478921 31369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:53243
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:05:43.514631 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37431
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:43.861105 31564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:43.861443 31564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:43.861624 31564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:43.869539 31564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:43.869659 31564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:43.869837 31564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:05:43.880507 31564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:43.881928 31564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:43.883460 31564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:43.898123 31569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:43.898051 31570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:43.898073 31572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:43.899029 31564 server_base.cc:1034] running on GCE node
I20260929 16:05:43.899765 31564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:43.900956 31564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:43.902179 31564 hybrid_clock.cc:648] HybridClock initialized: now 1790697943902120 us; error 50 us; skew 500 ppm
I20260929 16:05:43.902585 31564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:43 dist-test-slave-x07d krb5kdc[31324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:43.906284 31564 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:05:43.908071 31564 webserver.cc:460] Webserver started at http://127.20.121.130:35529/ using document root <none> and password file <none>
I20260929 16:05:43.908818 31564 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:43.908981 31564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:43.909288 31564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:43.912415 31564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "d1b07d11bea74b73aa122e33cb3849fb"
format_stamp: "Formatted at 2026-09-29 16:05:43 on dist-test-slave-x07d"
server_key: "9cfbbfda96ea039518c61fc3994d7138"
server_key_iv: "2a0ffade5e7932e5fc564bca5dc937aa"
server_key_version: "encryptionkey@0"
I20260929 16:05:43.913298 31564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "d1b07d11bea74b73aa122e33cb3849fb"
format_stamp: "Formatted at 2026-09-29 16:05:43 on dist-test-slave-x07d"
server_key: "9cfbbfda96ea039518c61fc3994d7138"
server_key_iv: "2a0ffade5e7932e5fc564bca5dc937aa"
server_key_version: "encryptionkey@0"
I20260929 16:05:43.919310 31564 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 16:05:43.923288 31579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:43.925134 31564 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260929 16:05:43.925369 31564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "d1b07d11bea74b73aa122e33cb3849fb"
format_stamp: "Formatted at 2026-09-29 16:05:43 on dist-test-slave-x07d"
server_key: "9cfbbfda96ea039518c61fc3994d7138"
server_key_iv: "2a0ffade5e7932e5fc564bca5dc937aa"
server_key_version: "encryptionkey@0"
I20260929 16:05:43.925614 31564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:43.939360 31564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:43.944205 31564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:43.944561 31564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:43.945624 31564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:43.947153 31564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:43.947266 31564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:43.947422 31564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:43.947499 31564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:43.983672 31564 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:41549
I20260929 16:05:43.983672 31691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:41549 every 8 connection(s)
I20260929 16:05:43.985167 31564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:05:43.988916 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 31564
I20260929 16:05:43.989121 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:05:43.989657 20966 external_mini_cluster.cc:1428] Setting key b6d195f0bcc029bf32ec35e9b3675b12
Sep 29 16:05:43 dist-test-slave-x07d krb5kdc[31324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697943, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:05:44.010993 31694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:43.987904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:39259 (local address 127.20.121.190:37431)
0929 16:05:43.988596 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:43.988606 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:43.989590 (+ 984us) server_negotiation.cc:407] Connection header received
0929 16:05:43.991766 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:43.991771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:43.991863 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:43.991975 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:43.994429 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:43.995340 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:43.998156 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:43.998624 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:43.998744 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.003072 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:44.003082 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:44.003091 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:44.003143 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:44.006899 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:44.007871 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:44.007881 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:44.007887 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:44.008066 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:44.008633 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:44.008641 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:44.008647 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:44.009071 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:44.009238 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:44.010147 (+ 909us) server_negotiation.cc:299] Negotiation successful
0929 16:05:44.010438 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":303,"threads_started":1}
I20260929 16:05:44.011957 31695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:43.988580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37431 (local address 127.20.121.130:39259)
0929 16:05:43.989407 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:43.989454 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:43.991437 (+ 1983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:43.992297 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:43.992310 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:43.993158 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:43.994136 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:43.994167 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:43.995539 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:43.995547 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:43.997847 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:43.997876 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:43.998908 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:43.998918 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:43.999306 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.000309 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:44.000334 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:44.002763 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:44.007138 (+ 4375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:44.007147 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:44.007165 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:44.007683 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:44.008231 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:44.008236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:44.008239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:44.008487 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:44.009227 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:44.009234 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:44.009641 (+ 407us) client_negotiation.cc:769] Sending connection context
0929 16:05:44.009970 (+ 329us) client_negotiation.cc:240] Negotiation successful
0929 16:05:44.010523 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":338,"threads_started":1}
I20260929 16:05:44.014178 31692 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37431
I20260929 16:05:44.014530 31692 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:44.015332 31692 heartbeater.cc:511] Master 127.20.121.190:37431 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
I20260929 16:05:44.017067 31369 ts_manager.cc:194] Registered new tserver with Master: d1b07d11bea74b73aa122e33cb3849fb (127.20.121.130:41549)
I20260929 16:05:44.018088 31369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:39259
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:05:44.050518 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37431
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:05:44.400326 31698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:05:44.400668 31698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:05:44.400820 31698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:05:44.408707 31698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:05:44.408862 31698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:05:44.409066 31698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:05:44.420926 31698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:05:44.422657 31698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:05:44.424233 31698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:05:44.439829 31704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:44.439829 31703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:05:44.440107 31706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:05:44.440879 31698 server_base.cc:1034] running on GCE node
I20260929 16:05:44.441599 31698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:05:44.442850 31698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:05:44.444222 31698 hybrid_clock.cc:648] HybridClock initialized: now 1790697944444185 us; error 147 us; skew 500 ppm
I20260929 16:05:44.444775 31698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:05:44 dist-test-slave-x07d krb5kdc[31324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:05:44.448724 31698 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:05:44.450541 31698 webserver.cc:460] Webserver started at http://127.20.121.131:40467/ using document root <none> and password file <none>
I20260929 16:05:44.451310 31698 fs_manager.cc:359] Metadata directory not provided
I20260929 16:05:44.451465 31698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:05:44.451777 31698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:05:44.455166 31698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3"
format_stamp: "Formatted at 2026-09-29 16:05:44 on dist-test-slave-x07d"
server_key: "eab6e77116a9fb1689f1f04afe138f7b"
server_key_iv: "6ff3f25067c63f5fcb9a43af6c7bae1c"
server_key_version: "encryptionkey@0"
I20260929 16:05:44.456108 31698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3"
format_stamp: "Formatted at 2026-09-29 16:05:44 on dist-test-slave-x07d"
server_key: "eab6e77116a9fb1689f1f04afe138f7b"
server_key_iv: "6ff3f25067c63f5fcb9a43af6c7bae1c"
server_key_version: "encryptionkey@0"
I20260929 16:05:44.462424 31698 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 16:05:44.467140 31713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:44.469099 31698 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260929 16:05:44.469327 31698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3"
format_stamp: "Formatted at 2026-09-29 16:05:44 on dist-test-slave-x07d"
server_key: "eab6e77116a9fb1689f1f04afe138f7b"
server_key_iv: "6ff3f25067c63f5fcb9a43af6c7bae1c"
server_key_version: "encryptionkey@0"
I20260929 16:05:44.469561 31698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:05:44.483192 31558 heartbeater.cc:503] Master 127.20.121.190:37431 was elected leader, sending a full tablet report...
I20260929 16:05:44.488056 31698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:05:44.492931 31698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:05:44.493255 31698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:05:44.494410 31698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:05:44.495882 31698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:05:44.495999 31698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:44.496152 31698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:05:44.496234 31698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:05:44.532958 31698 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:45117
I20260929 16:05:44.533021 31825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:45117 every 8 connection(s)
I20260929 16:05:44.534329 31698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:05:44.535432 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 31698
I20260929 16:05:44.535609 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:05:44.536177 20966 external_mini_cluster.cc:1428] Setting key c09ccd5b3c83d13ca3dbda60d439a551
Sep 29 16:05:44 dist-test-slave-x07d krb5kdc[31324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697944, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:05:44.556994 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.537333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:48037 (local address 127.20.121.190:37431)
0929 16:05:44.540344 (+ 3011us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:44.540349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:44.540367 (+ 18us) server_negotiation.cc:407] Connection header received
0929 16:05:44.541375 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:44.541381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:44.541466 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:44.541570 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:44.543210 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.544079 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.545652 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.546133 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.546236 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.549990 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:44.550000 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:44.550037 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:44.550090 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:44.553577 (+ 3487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:44.554456 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:44.554460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:44.554463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:44.554588 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:44.555057 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:44.555062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:44.555064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:44.555395 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:44.555534 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:44.556444 (+ 910us) server_negotiation.cc:299] Negotiation successful
0929 16:05:44.556629 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2895,"thread_start_us":2780,"threads_started":1}
I20260929 16:05:44.558147 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.538288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37431 (local address 127.20.121.131:48037)
0929 16:05:44.539418 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:44.539461 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:44.541077 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:44.541776 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:44.541784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:44.542436 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:05:44.543030 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.543047 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.544227 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.544231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.545488 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.545499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.546325 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.546330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.546524 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.547240 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:05:44.547263 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:05:44.549655 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:05:44.553728 (+ 4073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:44.553737 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:44.553751 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:44.554343 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:44.554702 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:05:44.554705 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:05:44.554707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:05:44.554898 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:05:44.555547 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:05:44.555555 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:05:44.556033 (+ 478us) client_negotiation.cc:769] Sending connection context
0929 16:05:44.556301 (+ 268us) client_negotiation.cc:240] Negotiation successful
0929 16:05:44.556775 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":420,"threads_started":1}
I20260929 16:05:44.559808 31826 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37431
I20260929 16:05:44.560247 31826 heartbeater.cc:464] Registering TS with master...
I20260929 16:05:44.561125 31826 heartbeater.cc:511] Master 127.20.121.190:37431 requested a full tablet report, sending...
I20260929 16:05:44.563167 31369 ts_manager.cc:194] Registered new tserver with Master: c6a6cdcf2ba64b5e8a8b27e20a743df3 (127.20.121.131:45117)
I20260929 16:05:44.564198 31369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:48037
I20260929 16:05:44.564878 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:05:44.584642 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.570597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36420 (local address 127.20.121.190:37431)
0929 16:05:44.570843 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:44.570850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:44.571207 (+ 357us) server_negotiation.cc:407] Connection header received
0929 16:05:44.571363 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:44.571367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:44.571443 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:44.571562 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:44.573106 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.573954 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.575570 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.575945 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.576048 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.578531 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:44.578543 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:44.578546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:44.578579 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:44.581433 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:44.582295 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:44.582300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:44.582302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:44.582421 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:44.582860 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:44.582863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:44.582865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:44.583227 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:44.583425 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:44.584032 (+ 607us) server_negotiation.cc:299] Negotiation successful
0929 16:05:44.584304 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 16:05:44.589233 31369 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:05:44.596558 31369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:05:44.634462 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.610843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51890 (local address 127.20.121.131:45117)
0929 16:05:44.623600 (+ 12757us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:44.623610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:44.623652 (+ 42us) server_negotiation.cc:407] Connection header received
0929 16:05:44.624241 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:44.624251 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:44.624622 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:44.624875 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:44.626513 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.628625 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.631397 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.633057 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.633244 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.633423 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:44.633854 (+ 431us) server_negotiation.cc:299] Negotiation successful
0929 16:05:44.634067 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12544,"thread_start_us":12356,"threads_started":1}
I20260929 16:05:44.634528 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.610640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:45117 (local address 127.0.0.1:51890)
0929 16:05:44.622975 (+ 12335us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:44.623447 (+ 472us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:44.623742 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:44.625275 (+ 1533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:44.625287 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:44.625352 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:44.626182 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.626206 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.628807 (+ 2601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.628812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.631228 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.631241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.633331 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.633339 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.633573 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.633646 (+ 73us) client_negotiation.cc:769] Sending connection context
0929 16:05:44.633795 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 16:05:44.634095 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11969,"mutex_wait_us":81,"spinlock_wait_cycles":4608,"thread_start_us":12913,"threads_started":1}
I20260929 16:05:44.636909 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.610968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41470 (local address 127.20.121.129:33315)
0929 16:05:44.621947 (+ 10979us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:44.621962 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:44.623989 (+ 2027us) server_negotiation.cc:407] Connection header received
0929 16:05:44.624292 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:44.624301 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:44.624622 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:44.624839 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:44.626583 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.628241 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.633566 (+ 5325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.635390 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.635606 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.635739 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:44.636311 (+ 572us) server_negotiation.cc:299] Negotiation successful
0929 16:05:44.636505 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10709,"thread_start_us":11013,"threads_started":1}
I20260929 16:05:44.638000 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.610839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:33315 (local address 127.0.0.1:41470)
0929 16:05:44.623873 (+ 13034us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:44.623890 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:44.624096 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:44.625276 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:44.625288 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:44.625361 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:44.626293 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.626311 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.628426 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.628430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.631226 (+ 2796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.631241 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.635902 (+ 4661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.635908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.636090 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.636124 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 16:05:44.637391 (+ 1267us) client_negotiation.cc:240] Negotiation successful
0929 16:05:44.637557 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12902,"mutex_wait_us":92,"spinlock_wait_cycles":15360,"thread_start_us":11618,"threads_started":1}
I20260929 16:05:44.638162 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.624254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59874 (local address 127.20.121.130:41549)
0929 16:05:44.627855 (+ 3601us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:44.627864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:44.629691 (+ 1827us) server_negotiation.cc:407] Connection header received
0929 16:05:44.629799 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:44.629808 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:44.630130 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:44.630336 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:44.631155 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.632701 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.635361 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.636818 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.636986 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.637181 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:44.637670 (+ 489us) server_negotiation.cc:299] Negotiation successful
0929 16:05:44.637837 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3427,"thread_start_us":3321,"threads_started":1}
I20260929 16:05:44.638180 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.624025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:41549 (local address 127.0.0.1:59874)
0929 16:05:44.629317 (+ 5292us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:44.629335 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:44.629482 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:44.630416 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:44.630422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:44.630452 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:44.630922 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.630998 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.632870 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.632875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.635177 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.635194 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.637312 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.637317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.637490 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.637517 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 16:05:44.637624 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 16:05:44.637760 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5155,"thread_start_us":4969,"threads_started":1}
I20260929 16:05:44.639333 31761 tablet_service.cc:1467] Processing CreateTablet for tablet 96ad9a6fec5741d08767ba5da48b400d (DEFAULT_TABLE table=test-table [id=2f536fcd5e834c66b6abab04c689dcac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:44.640818 31761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ad9a6fec5741d08767ba5da48b400d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:44.642100 31493 tablet_service.cc:1467] Processing CreateTablet for tablet 96ad9a6fec5741d08767ba5da48b400d (DEFAULT_TABLE table=test-table [id=2f536fcd5e834c66b6abab04c689dcac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:44.642099 31627 tablet_service.cc:1467] Processing CreateTablet for tablet 96ad9a6fec5741d08767ba5da48b400d (DEFAULT_TABLE table=test-table [id=2f536fcd5e834c66b6abab04c689dcac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:05:44.643529 31493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ad9a6fec5741d08767ba5da48b400d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:44.643539 31627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ad9a6fec5741d08767ba5da48b400d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:05:44.654259 31845 tablet_bootstrap.cc:492] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: Bootstrap starting.
I20260929 16:05:44.657627 31845 tablet_bootstrap.cc:654] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:44.658839 31845 log.cc:824] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:44.661526 31847 tablet_bootstrap.cc:492] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: Bootstrap starting.
I20260929 16:05:44.659844 31846 tablet_bootstrap.cc:492] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: Bootstrap starting.
I20260929 16:05:44.663902 31845 tablet_bootstrap.cc:492] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: No bootstrap required, opened a new log
I20260929 16:05:44.664242 31845 ts_tablet_manager.cc:1397] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260929 16:05:44.665606 31846 tablet_bootstrap.cc:654] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:44.665606 31847 tablet_bootstrap.cc:654] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: Neither blocks nor log segments found. Creating new log.
I20260929 16:05:44.667167 31847 log.cc:824] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:44.667165 31846 log.cc:824] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: Log is configured to *not* fsync() on all Append() calls
I20260929 16:05:44.670974 31847 tablet_bootstrap.cc:492] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: No bootstrap required, opened a new log
I20260929 16:05:44.671283 31847 ts_tablet_manager.cc:1397] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260929 16:05:44.671386 31846 tablet_bootstrap.cc:492] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: No bootstrap required, opened a new log
I20260929 16:05:44.671653 31846 ts_tablet_manager.cc:1397] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260929 16:05:44.673245 31845 raft_consensus.cc:348] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 } } peers { permanent_uuid: "212719921ce346fcbeb7a8a890be2e0e" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 33315 } } peers { permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 } }
I20260929 16:05:44.673781 31845 raft_consensus.cc:374] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:44.673913 31845 raft_consensus.cc:729] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a6cdcf2ba64b5e8a8b27e20a743df3, State: Initialized, Role: FOLLOWER
I20260929 16:05:44.674642 31845 consensus_queue.cc:260] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 } } peers { permanent_uuid: "212719921ce346fcbeb7a8a890be2e0e" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 33315 } } peers { permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 } }
I20260929 16:05:44.675760 31826 heartbeater.cc:503] Master 127.20.121.190:37431 was elected leader, sending a full tablet report...
I20260929 16:05:44.676532 31845 ts_tablet_manager.cc:1428] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260929 16:05:44.682631 31846 raft_consensus.cc:348] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 } } peers { permanent_uuid: "212719921ce346fcbeb7a8a890be2e0e" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 33315 } } peers { permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 } }
I20260929 16:05:44.682632 31847 raft_consensus.cc:348] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 } } peers { permanent_uuid: "212719921ce346fcbeb7a8a890be2e0e" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 33315 } } peers { permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 } }
I20260929 16:05:44.683215 31847 raft_consensus.cc:374] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:44.683214 31846 raft_consensus.cc:374] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:05:44.683327 31847 raft_consensus.cc:729] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 212719921ce346fcbeb7a8a890be2e0e, State: Initialized, Role: FOLLOWER
I20260929 16:05:44.683327 31846 raft_consensus.cc:729] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b07d11bea74b73aa122e33cb3849fb, State: Initialized, Role: FOLLOWER
I20260929 16:05:44.684028 31846 consensus_queue.cc:260] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 } } peers { permanent_uuid: "212719921ce346fcbeb7a8a890be2e0e" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 33315 } } peers { permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 } }
I20260929 16:05:44.684017 31847 consensus_queue.cc:260] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 } } peers { permanent_uuid: "212719921ce346fcbeb7a8a890be2e0e" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 33315 } } peers { permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 } }
I20260929 16:05:44.685175 31692 heartbeater.cc:503] Master 127.20.121.190:37431 was elected leader, sending a full tablet report...
I20260929 16:05:44.686163 31846 ts_tablet_manager.cc:1428] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260929 16:05:44.686264 31847 ts_tablet_manager.cc:1428] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260929 16:05:44.699380 31559 tablet.cc:2369] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:05:44.737499 31693 tablet.cc:2369] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:05:44.786214 31827 tablet.cc:2369] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:05:44.888914 31853 raft_consensus.cc:482] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:05:44.889272 31853 raft_consensus.cc:504] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 } } peers { permanent_uuid: "212719921ce346fcbeb7a8a890be2e0e" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 33315 } } peers { permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 } }
I20260929 16:05:44.891299 31853 leader_election.cc:288] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6a6cdcf2ba64b5e8a8b27e20a743df3 (127.20.121.131:45117), d1b07d11bea74b73aa122e33cb3849fb (127.20.121.130:41549)
I20260929 16:05:44.901010 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.891728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:47891 (local address 127.20.121.131:45117)
0929 16:05:44.891984 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:44.891995 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:44.892573 (+ 578us) server_negotiation.cc:407] Connection header received
0929 16:05:44.892842 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:44.892851 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:44.892966 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:44.893114 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:44.893915 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.895261 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.898161 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.899607 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.899847 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.900126 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:44.900564 (+ 438us) server_negotiation.cc:299] Negotiation successful
0929 16:05:44.900736 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 16:05:44.901125 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.893072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:60795 (local address 127.20.121.130:41549)
0929 16:05:44.893269 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:44.893276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:44.893751 (+ 475us) server_negotiation.cc:407] Connection header received
0929 16:05:44.893921 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:44.893927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:44.894009 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:44.894140 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:05:44.894919 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.895901 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.898339 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.899973 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.900209 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.900300 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:44.900736 (+ 436us) server_negotiation.cc:299] Negotiation successful
0929 16:05:44.900880 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 16:05:44.901160 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.891718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:45117 (local address 127.20.121.129:47891)
0929 16:05:44.892438 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:44.892456 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:44.892659 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:44.893224 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:44.893235 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:44.893271 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:44.893767 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.893781 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.895399 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.895403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.897783 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.897800 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.900068 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.900079 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.900341 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.900381 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 16:05:44.900522 (+ 141us) client_negotiation.cc:240] Negotiation successful
0929 16:05:44.900734 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":1,"thread_start_us":427,"threads_started":1}
I20260929 16:05:44.901842 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.892715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:41549 (local address 127.20.121.129:60795)
0929 16:05:44.893635 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0929 16:05:44.893653 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:05:44.893790 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:05:44.894195 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:05:44.894204 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:05:44.894266 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:05:44.894723 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.894737 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.896022 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.896028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.898161 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:05:44.898172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.900324 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.900332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:05:44.900524 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.900564 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 16:05:44.901373 (+ 809us) client_negotiation.cc:240] Negotiation successful
0929 16:05:44.901594 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":553,"threads_started":1}
I20260929 16:05:44.902591 31781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ad9a6fec5741d08767ba5da48b400d" candidate_uuid: "212719921ce346fcbeb7a8a890be2e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" is_pre_election: true
I20260929 16:05:44.902843 31647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ad9a6fec5741d08767ba5da48b400d" candidate_uuid: "212719921ce346fcbeb7a8a890be2e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b07d11bea74b73aa122e33cb3849fb" is_pre_election: true
I20260929 16:05:44.903143 31781 raft_consensus.cc:2454] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 212719921ce346fcbeb7a8a890be2e0e in term 0.
I20260929 16:05:44.903422 31647 raft_consensus.cc:2454] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 212719921ce346fcbeb7a8a890be2e0e in term 0.
I20260929 16:05:44.903885 31449 leader_election.cc:302] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 212719921ce346fcbeb7a8a890be2e0e, c6a6cdcf2ba64b5e8a8b27e20a743df3; no voters:
I20260929 16:05:44.904292 31853 raft_consensus.cc:2781] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:05:44.904454 31853 raft_consensus.cc:482] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:05:44.904567 31853 raft_consensus.cc:3037] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:44.906862 31853 raft_consensus.cc:504] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 } } peers { permanent_uuid: "212719921ce346fcbeb7a8a890be2e0e" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 33315 } } peers { permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 } }
I20260929 16:05:44.907512 31853 leader_election.cc:288] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [CANDIDATE]: Term 1 election: Requested vote from peers c6a6cdcf2ba64b5e8a8b27e20a743df3 (127.20.121.131:45117), d1b07d11bea74b73aa122e33cb3849fb (127.20.121.130:41549)
I20260929 16:05:44.908061 31781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ad9a6fec5741d08767ba5da48b400d" candidate_uuid: "212719921ce346fcbeb7a8a890be2e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3"
I20260929 16:05:44.908113 31647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ad9a6fec5741d08767ba5da48b400d" candidate_uuid: "212719921ce346fcbeb7a8a890be2e0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b07d11bea74b73aa122e33cb3849fb"
I20260929 16:05:44.908265 31781 raft_consensus.cc:3037] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:44.908380 31647 raft_consensus.cc:3037] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:05:44.910754 31781 raft_consensus.cc:2454] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 212719921ce346fcbeb7a8a890be2e0e in term 1.
I20260929 16:05:44.911110 31647 raft_consensus.cc:2454] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 212719921ce346fcbeb7a8a890be2e0e in term 1.
I20260929 16:05:44.911413 31449 leader_election.cc:302] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 212719921ce346fcbeb7a8a890be2e0e, c6a6cdcf2ba64b5e8a8b27e20a743df3; no voters:
I20260929 16:05:44.911792 31853 raft_consensus.cc:2781] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:05:44.912267 31853 raft_consensus.cc:686] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 1 LEADER]: Becoming Leader. State: Replica: 212719921ce346fcbeb7a8a890be2e0e, State: Running, Role: LEADER
I20260929 16:05:44.912794 31853 consensus_queue.cc:237] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 } } peers { permanent_uuid: "212719921ce346fcbeb7a8a890be2e0e" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 33315 } } peers { permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 } }
I20260929 16:05:44.917810 31369 catalog_manager.cc:5495] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e reported cstate change: term changed from 0 to 1, leader changed from <none> to 212719921ce346fcbeb7a8a890be2e0e (127.20.121.129). New cstate: current_term: 1 leader_uuid: "212719921ce346fcbeb7a8a890be2e0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "212719921ce346fcbeb7a8a890be2e0e" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 33315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 } health_report { overall_health: UNKNOWN } } }
I20260929 16:05:44.943876 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:44.936545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41486 (local address 127.20.121.129:33315)
0929 16:05:44.936941 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:44.936947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:44.936972 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:05:44.937439 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:44.937445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:44.937552 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:44.937783 (+ 231us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:05:44.938386 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.939363 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.940861 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:44.941382 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:44.941523 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:44.942204 (+ 681us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:05:44.942995 (+ 791us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:05:44.943162 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:44.943355 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 16:05:44.943492 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199}
I20260929 16:05:44.954700 31647 raft_consensus.cc:1261] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb [term 1 FOLLOWER]: Refusing update from remote peer 212719921ce346fcbeb7a8a890be2e0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:44.954834 31781 raft_consensus.cc:1261] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3 [term 1 FOLLOWER]: Refusing update from remote peer 212719921ce346fcbeb7a8a890be2e0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:05:44.956156 31856 consensus_queue.cc:1035] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b07d11bea74b73aa122e33cb3849fb" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:44.956681 31853 consensus_queue.cc:1035] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6a6cdcf2ba64b5e8a8b27e20a743df3" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:05:44.993368 31368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:44.993824 31368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:05:44.999652 31368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0cbd466ebdf4a60b716a08a0095a365: Sending DeleteTablet for 3 replicas of tablet 96ad9a6fec5741d08767ba5da48b400d
I20260929 16:05:45.001417 31761 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ad9a6fec5741d08767ba5da48b400d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:44 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:51890
I20260929 16:05:45.001598 31493 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ad9a6fec5741d08767ba5da48b400d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:44 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:41470
I20260929 16:05:45.006726 31627 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ad9a6fec5741d08767ba5da48b400d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:05:44 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:59874
I20260929 16:05:45.009534 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 31430
I20260929 16:05:45.012081 31869 tablet_replica.cc:330] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: stopping tablet replica
I20260929 16:05:45.012360 31870 tablet_replica.cc:330] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: stopping tablet replica
I20260929 16:05:45.012676 31869 raft_consensus.cc:2229] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:45.012894 31870 raft_consensus.cc:2229] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:05:45.013434 31869 raft_consensus.cc:2258] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:45.013715 31870 raft_consensus.cc:2258] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:45.015511 31871 tablet_replica.cc:330] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: stopping tablet replica
I20260929 16:05:45.016033 31871 raft_consensus.cc:2229] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:05:45.016233 31869 ts_tablet_manager.cc:1905] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:45.016633 31871 raft_consensus.cc:2258] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:05:45.018817 31871 ts_tablet_manager.cc:1905] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:45.020870 31870 ts_tablet_manager.cc:1905] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:05:45.025027 31869 ts_tablet_manager.cc:1918] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:45.025311 31869 log.cc:1196] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/96ad9a6fec5741d08767ba5da48b400d
I20260929 16:05:45.026124 31869 ts_tablet_manager.cc:1939] T 96ad9a6fec5741d08767ba5da48b400d P c6a6cdcf2ba64b5e8a8b27e20a743df3: Deleting consensus metadata
I20260929 16:05:45.028432 31356 catalog_manager.cc:4841] TS c6a6cdcf2ba64b5e8a8b27e20a743df3 (127.20.121.131:45117): tablet 96ad9a6fec5741d08767ba5da48b400d (table test-table [id=2f536fcd5e834c66b6abab04c689dcac]) successfully deleted
I20260929 16:05:45.029471 31871 ts_tablet_manager.cc:1918] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:45.029685 31871 log.cc:1196] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/96ad9a6fec5741d08767ba5da48b400d
I20260929 16:05:45.030376 31871 ts_tablet_manager.cc:1939] T 96ad9a6fec5741d08767ba5da48b400d P d1b07d11bea74b73aa122e33cb3849fb: Deleting consensus metadata
Sep 29 16:05:45 dist-test-slave-x07d krb5kdc[31324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:05:45.032604 31356 catalog_manager.cc:4841] TS d1b07d11bea74b73aa122e33cb3849fb (127.20.121.130:41549): tablet 96ad9a6fec5741d08767ba5da48b400d (table test-table [id=2f536fcd5e834c66b6abab04c689dcac]) successfully deleted
I20260929 16:05:45.037209 31870 ts_tablet_manager.cc:1918] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:05:45.037459 31870 log.cc:1196] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/96ad9a6fec5741d08767ba5da48b400d
I20260929 16:05:45.038130 31870 ts_tablet_manager.cc:1939] T 96ad9a6fec5741d08767ba5da48b400d P 212719921ce346fcbeb7a8a890be2e0e: Deleting consensus metadata
I20260929 16:05:45.040429 31353 catalog_manager.cc:4841] TS 212719921ce346fcbeb7a8a890be2e0e (127.20.121.129:33315): tablet 96ad9a6fec5741d08767ba5da48b400d (table test-table [id=2f536fcd5e834c66b6abab04c689dcac]) successfully deleted
I20260929 16:05:45.041600 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:45.011466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41500 (local address 127.20.121.129:33315)
0929 16:05:45.011730 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:45.011737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:45.015100 (+ 3363us) server_negotiation.cc:407] Connection header received
0929 16:05:45.015200 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:45.015209 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:45.015358 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:45.015499 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:45.017916 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:45.019202 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:45.021638 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:45.022509 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:45.022654 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:45.033047 (+ 10393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:45.033069 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:45.033096 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:45.033200 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:45.037814 (+ 4614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:45.038850 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:45.038861 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:45.038866 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:45.039050 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:45.039487 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:45.039493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:45.039496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:45.040023 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:45.040387 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:45.040989 (+ 602us) server_negotiation.cc:299] Negotiation successful
0929 16:05:45.041242 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260929 16:05:45.042639 31553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:45.162386 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 31430
I20260929 16:05:45.185279 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 31564
Sep 29 16:05:45 dist-test-slave-x07d krb5kdc[31324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:05:45.204066 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:45.186378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.20.121.130:41549)
0929 16:05:45.186592 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:45.186596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:45.186620 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:05:45.187061 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:45.187067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:45.187141 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:45.187265 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:45.189127 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:45.189848 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:45.191400 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:45.191825 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:45.191935 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:45.196851 (+ 4916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:45.196868 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:45.196937 (+ 69us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:45.196988 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:45.200440 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:45.201598 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:45.201606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:45.201608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:45.201778 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:45.202371 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:45.202375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:45.202378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:45.202795 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:45.203056 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:45.203452 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 16:05:45.203642 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 16:05:45.204819 31687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:45.301999 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 31564
I20260929 16:05:45.323096 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 31698
Sep 29 16:05:45 dist-test-slave-x07d krb5kdc[31324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:05:45.343323 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:05:45.324154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51904 (local address 127.20.121.131:45117)
0929 16:05:45.324418 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0929 16:05:45.324424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:05:45.324448 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:05:45.324547 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:05:45.324570 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:05:45.324664 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:05:45.324761 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:05:45.326538 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:45.327261 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:45.328724 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:05:45.329160 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:05:45.329288 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:05:45.335279 (+ 5991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:05:45.335297 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:05:45.335315 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:05:45.335374 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:05:45.339057 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:45.340509 (+ 1452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:45.340519 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:45.340523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:45.340742 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:05:45.341302 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:05:45.341306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:05:45.341308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:05:45.341733 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:05:45.342122 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0929 16:05:45.342673 (+ 551us) server_negotiation.cc:299] Negotiation successful
0929 16:05:45.342941 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260929 16:05:45.344087 31821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:45.438778 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 31698
I20260929 16:05:45.457199 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 31337
I20260929 16:05:45.458792 31399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:05:45.590493 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 31337
2026-09-29T16:05:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4640 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:05:45 dist-test-slave-x07d krb5kdc[31879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:05:45 dist-test-slave-x07d krb5kdc[31879](info): set up 2 sockets
Sep 29 16:05:45 dist-test-slave-x07d krb5kdc[31879](info): commencing operation
krb5kdc: starting...
W20260929 16:05:46.787070 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:05:46 dist-test-slave-x07d krb5kdc[31879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:05:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:05:46Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.121.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.121.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/rangeradmin_127.20.121.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/rangeradmin_127.20.121.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.121.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.121.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/rangerlookup_127.20.121.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/rangerlookup_127.20.121.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.121.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.121.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/HTTP_127.20.121.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/HTTP_127.20.121.148@KRBTEST.COM.keytab.
I20260929 16:05:47.167099 20966 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDT3jwB/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/postgres -l logfile start
2026-09-29 16:05:50.206 UTC [31909] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 16:05:50.206 UTC [31909] LOG: listening on IPv4 address "127.20.121.148", port 56647
2026-09-29 16:05:50.212 UTC [31909] LOG: listening on Unix socket "/tmp/.s.PGSQL.56647"
2026-09-29 16:05:50.244 UTC [31912] LOG: database system was shut down at 2026-09-29 16:05:48 UTC
2026-09-29 16:05:50.249 UTC [31909] LOG: database system is ready to accept connections
I20260929 16:05:51.242745 20966 mini_postgres.cc:96] Postgres bound to 56647
2026-09-29 16:05:51.261 UTC [31920] FATAL: database "slave" does not exist
127.20.121.148:56647 - accepting connections
I20260929 16:05:51.262162 20966 mini_ranger.cc:161] Starting Ranger...
I20260929 16:05:51.291167 20966 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 16:05:51.963095 20966 mini_ranger.cc:87] Created ranger Postgres database
I20260929 16:05:51.963294 20966 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-admin
2026-09-29 16:05:52,290 [I] DB FLAVOR :POSTGRES
2026-09-29 16:05:52,291 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 16:05:52,291 [I] Checking connection..
2026-09-29 16:05:52,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 16:05:52,537 [I] Checking connection passed.
2026-09-29 16:05:52,537 [I] --------- Verifying version history table ---------
2026-09-29 16:05:52,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 16:05:52,822 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 16:05:52,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 16:05:53,075 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 16:05:53,076 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 16:05:53,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 16:05:53.346 UTC [31984] WARNING: there is no transaction in progress
2026-09-29 16:05:53,353 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 16:05:53,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 16:05:53,625 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 16:05:53,626 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 16:05:53,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 16:05:53,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 16:05:54,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x07d.c.gcp-upstream.internal','N') ;"
2026-09-29 16:05:54,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 16:05:54,690 [I] Table x_portal_user does not exist in database ranger
2026-09-29 16:05:54,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 16:05:54,953 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 16:05:54,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 16:05:55,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 16:05:55,484 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 16:05:55,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 16:05:56.576 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.595 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.616 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.635 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.658 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.686 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.754 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.764 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.781 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.794 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.805 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.822 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.834 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.842 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.850 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:56.859 UTC [32092] WARNING: there is no transaction in progress
2026-09-29 16:05:57,697 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 16:05:57,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 16:05:57,955 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 16:05:57,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 16:05:58,204 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 16:05:58,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 16:05:58,467 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 16:05:58,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 16:05:58,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 16:05:58,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x07d.c.gcp-upstream.internal';"
2026-09-29 16:05:59,232 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 16:05:59,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 16:05:59,497 [I] DB_PATCHES have already been applied
I20260929 16:05:59.503278 20966 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDT3jwB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 16:05:59.503444 20966 mini_ranger.cc:193] Using host: 127.20.121.148
I20260929 16:05:59.519562 20966 mini_ranger.cc:227] Ranger admin URL: http://127.20.121.148:50139
Sep 29, 2026 4:05:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 4:05:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 4:05:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 4:05:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 4:05:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 4:06:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 4:06:00 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 16:06:00.756616 20966 mini_ranger.cc:160] Time spent starting Ranger: real 9.494s user 0.002s sys 0.041s
Sep 29, 2026 4:06:21 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 4:06:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 4:06:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 4:06:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 4:06:22 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 16:06:23.007408 20966 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.121.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.121.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/rangerkms_127.20.121.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/rangerkms_127.20.121.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.121.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.121.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.121.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/HTTP_127.20.121.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.121.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/HTTP_127.20.121.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 16:06:23 dist-test-slave-x07d krb5kdc[31879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697983, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 16:06:23.206991 20966 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 16:06:23.232609 20966 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 16:06:24.412923 20966 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 16:06:24.413036 20966 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-kms
I20260929 16:06:24.413204 20966 mini_ranger_kms.cc:227] Using postgres at 127.20.121.148:56647
2026-09-29 16:06:24,864 [I] DB FLAVOR :POSTGRES
2026-09-29 16:06:24,865 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 16:06:24,865 [I] Checking connection
2026-09-29 16:06:24,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 16:06:25,121 [I] connection success
2026-09-29 16:06:25,121 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 16:06:25,121 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 16:06:25,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 16:06:25,375 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 16:06:25,376 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 16:06:25,376 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 16:06:25,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:56647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 16:06:25,696 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 16:06:26.329449 20966 mini_ranger_kms.cc:310] Created kms service
I20260929 16:06:26.569083 20966 mini_ranger_kms.cc:326] Created kudu user
I20260929 16:06:26.623669 20966 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 16:06:26 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697983, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:06:27.091181 20966 mini_ranger_kms.cc:384] Added ranger policy
I20260929 16:06:27.091552 20966 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 16:06:27.091598 20966 mini_ranger_kms.cc:240] Using host: 127.20.121.148
I20260929 16:06:27.114219 20966 mini_ranger_kms.cc:276] Ranger KMS PID: 32273
I20260929 16:06:27.114447 20966 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.121.148:36513
Sep 29, 2026 4:06:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 4:06:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 4:06:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 4:06:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 4:06:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 4:06:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 16:06:29.480247 20966 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.273s user 0.135s sys 0.342s
I20260929 16:06:29.480501 20966 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 16:06:29.480705 20966 mini_ranger_kms.cc:401] 127.20.121.148:36513/kms/v1/keys
Sep 29 16:06:29 dist-test-slave-x07d krb5kdc[31879](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790697989, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.121.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 16:06:30 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697989, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.121.148@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
Sep 29 16:06:30 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697989, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.121.148@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
Sep 29 16:06:32 dist-test-slave-x07d krb5kdc[31879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:06:33.050499 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34633
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:34633
--ranger_config_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:36513/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:06:33.429473 32323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:06:33.429847 32323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:06:33.430032 32323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:06:33.437494 32323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:06:33.437671 32323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:06:33.437789 32323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:06:33.437884 32323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:06:33.437983 32323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:06:33.449292 32323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.121.148:36513/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:34633
--ranger_config_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:06:33.450870 32323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:06:33.452780 32323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:06:33.467257 32328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:06:33.467188 32331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:06:33.467135 32329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:06:33.468230 32323 server_base.cc:1034] running on GCE node
I20260929 16:06:33.469085 32323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:06:33.470770 32323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:06:33.472025 32323 hybrid_clock.cc:648] HybridClock initialized: now 1790697993471975 us; error 41 us; skew 500 ppm
I20260929 16:06:33.472522 32323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:06:33 dist-test-slave-x07d krb5kdc[31879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697993, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:06:33.480675 32323 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:06:33.482615 32323 webserver.cc:460] Webserver started at http://127.20.121.190:36967/ using document root <none> and password file <none>
I20260929 16:06:33.483376 32323 fs_manager.cc:359] Metadata directory not provided
I20260929 16:06:33.483489 32323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:06:33.483767 32323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 16:06:33 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697993, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:06:33.539240 32323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "9e3d25b91f284998bcc6e3783b61ee2f"
format_stamp: "Formatted at 2026-09-29 16:06:33 on dist-test-slave-x07d"
server_key: "8a8b548e25e4d2d55eeb2e3259cf4494"
server_key_iv: "4b415b99a4e4133a39a09d7ad7cf1e02"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:33.540243 32323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "9e3d25b91f284998bcc6e3783b61ee2f"
format_stamp: "Formatted at 2026-09-29 16:06:33 on dist-test-slave-x07d"
server_key: "8a8b548e25e4d2d55eeb2e3259cf4494"
server_key_iv: "4b415b99a4e4133a39a09d7ad7cf1e02"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:33.547120 32323 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260929 16:06:33.564014 32338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:06:33.565732 32323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260929 16:06:33.565954 32323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "9e3d25b91f284998bcc6e3783b61ee2f"
format_stamp: "Formatted at 2026-09-29 16:06:33 on dist-test-slave-x07d"
server_key: "8a8b548e25e4d2d55eeb2e3259cf4494"
server_key_iv: "4b415b99a4e4133a39a09d7ad7cf1e02"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:33.566254 32323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:06:33.599138 32323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:06:33.605023 32323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:06:33.605406 32323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:06:33.628656 32323 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:34633
I20260929 16:06:33.628679 32389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:34633 every 8 connection(s)
I20260929 16:06:33.630314 32323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:06:33.633539 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 32323
I20260929 16:06:33.633716 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:06:33.634508 20966 external_mini_cluster.cc:1428] Setting key a0a17ea40fcef8ff74c1041873e56ebe
I20260929 16:06:33.635082 32390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:06:33.649251 32390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f: Bootstrap starting.
Sep 29 16:06:33 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:06:33.654069 32390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f: Neither blocks nor log segments found. Creating new log.
I20260929 16:06:33.655295 32390 log.cc:824] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f: Log is configured to *not* fsync() on all Append() calls
I20260929 16:06:33.659641 32390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f: No bootstrap required, opened a new log
I20260929 16:06:33.660543 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:33.636502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57826 (local address 127.20.121.190:34633)
0929 16:06:33.637363 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:33.637379 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:33.637671 (+ 292us) server_negotiation.cc:407] Connection header received
0929 16:06:33.640187 (+ 2516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:33.640213 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:33.640649 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:33.641255 (+ 606us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:06:33.642726 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:33.644109 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:33.645442 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:33.646344 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:33.646578 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:33.651505 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:06:33.651519 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:06:33.651536 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:06:33.651599 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:06:33.655397 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:33.656543 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:33.656554 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:33.656566 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:33.656796 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:33.657268 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:33.657272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:33.657274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:33.657847 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:06:33.658169 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:33.658653 (+ 484us) server_negotiation.cc:299] Negotiation successful
0929 16:06:33.659101 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":379,"threads_started":1}
I20260929 16:06:33.668830 32390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3d25b91f284998bcc6e3783b61ee2f" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34633 } }
I20260929 16:06:33.669431 32390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:06:33.669551 32390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e3d25b91f284998bcc6e3783b61ee2f, State: Initialized, Role: FOLLOWER
I20260929 16:06:33.670437 32390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3d25b91f284998bcc6e3783b61ee2f" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34633 } }
I20260929 16:06:33.670745 32390 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:06:33.670918 32390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:06:33.671145 32390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:06:33.674135 32390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3d25b91f284998bcc6e3783b61ee2f" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34633 } }
I20260929 16:06:33.674746 32390 leader_election.cc:302] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e3d25b91f284998bcc6e3783b61ee2f; no voters:
I20260929 16:06:33.675388 32390 leader_election.cc:288] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:06:33.675565 32395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:06:33.676586 32395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [term 1 LEADER]: Becoming Leader. State: Replica: 9e3d25b91f284998bcc6e3783b61ee2f, State: Running, Role: LEADER
I20260929 16:06:33.677282 32395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3d25b91f284998bcc6e3783b61ee2f" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34633 } }
I20260929 16:06:33.678198 32390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:06:33.680323 32397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e3d25b91f284998bcc6e3783b61ee2f. Latest consensus state: current_term: 1 leader_uuid: "9e3d25b91f284998bcc6e3783b61ee2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3d25b91f284998bcc6e3783b61ee2f" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34633 } } }
I20260929 16:06:33.680874 32397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [sys.catalog]: This master's current role is: LEADER
I20260929 16:06:33.681063 32396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e3d25b91f284998bcc6e3783b61ee2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e3d25b91f284998bcc6e3783b61ee2f" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34633 } } }
I20260929 16:06:33.682659 32396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f [sys.catalog]: This master's current role is: LEADER
I20260929 16:06:33.688051 32400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:06:33.693909 32400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:06:33.694809 32390 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 16:06:33.714802 32400 catalog_manager.cc:1347] Generated new cluster ID: 939115b3f0b24d7e89f119f673bead27
I20260929 16:06:33.714991 32400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:06:33.730295 32400 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:06:33.732209 32400 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:06:33.745121 32400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f: Generated new TSK 0
I20260929 16:06:33.746567 32400 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:06:34.551739 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:36513/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34633
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:06:34 dist-test-slave-x07d krb5kdc[31879](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790697994, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 16:06:35.008538 32435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:06:35.009119 32435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:06:35.009328 32435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:06:35.021644 32435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:06:35.021818 32435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:06:35.022127 32435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:06:35.038918 32435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.121.148:36513/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:06:35.040951 32435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:06:35.042467 32435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:06:35.070545 32446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:06:35.073691 32445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:06:35.075285 32435 server_base.cc:1034] running on GCE node
W20260929 16:06:35.075773 32448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:06:35.076728 32435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:06:35.078143 32435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:06:35.079478 32435 hybrid_clock.cc:648] HybridClock initialized: now 1790697995079404 us; error 55 us; skew 500 ppm
I20260929 16:06:35.080013 32435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:06:35 dist-test-slave-x07d krb5kdc[31879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:06:35.084829 32435 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:06:35.087610 32435 webserver.cc:460] Webserver started at http://127.20.121.129:35995/ using document root <none> and password file <none>
I20260929 16:06:35.088703 32435 fs_manager.cc:359] Metadata directory not provided
I20260929 16:06:35.088874 32435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:06:35.089313 32435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 16:06:35 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:06:35.111306 32435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "97a48baaf8ad43109bfca58fc78c823d"
format_stamp: "Formatted at 2026-09-29 16:06:35 on dist-test-slave-x07d"
server_key: "961814f4c77dca7aa1334fd60440ea86"
server_key_iv: "3526785ca560dd3751cd090a6d5298e6"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:35.112607 32435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "97a48baaf8ad43109bfca58fc78c823d"
format_stamp: "Formatted at 2026-09-29 16:06:35 on dist-test-slave-x07d"
server_key: "961814f4c77dca7aa1334fd60440ea86"
server_key_iv: "3526785ca560dd3751cd090a6d5298e6"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:35.136511 32435 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.012s sys 0.008s
I20260929 16:06:35.171694 32455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:06:35.173933 32435 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.003s sys 0.007s
I20260929 16:06:35.174305 32435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "97a48baaf8ad43109bfca58fc78c823d"
format_stamp: "Formatted at 2026-09-29 16:06:35 on dist-test-slave-x07d"
server_key: "961814f4c77dca7aa1334fd60440ea86"
server_key_iv: "3526785ca560dd3751cd090a6d5298e6"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:35.174696 32435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:06:35.232710 32435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:06:35.239396 32435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:06:35.239892 32435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:06:35.245220 32435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:06:35.248293 32435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:06:35.248427 32435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:06:35.248703 32435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:06:35.248805 32435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:06:35 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697994, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:06:35.924048 32435 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:44989
I20260929 16:06:35.924117 32568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:44989 every 8 connection(s)
I20260929 16:06:35.925983 32435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:06:35.934696 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 32435
I20260929 16:06:35.934981 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:06:35.936036 20966 external_mini_cluster.cc:1428] Setting key bc323edeed57e0508b1965fc2e6ac0ac
Sep 29 16:06:35 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697994, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
Sep 29 16:06:35 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:06:35.984186 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:35.950910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:58575 (local address 127.20.121.190:34633)
0929 16:06:35.953080 (+ 2170us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:35.953086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:35.954586 (+ 1500us) server_negotiation.cc:407] Connection header received
0929 16:06:35.956639 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:35.956650 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:35.956767 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:35.956884 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:06:35.960552 (+ 3668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:35.961981 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:35.965179 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:35.966153 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:35.966298 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:35.973270 (+ 6972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:06:35.973279 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:06:35.973285 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:06:35.973339 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:06:35.977548 (+ 4209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:35.979126 (+ 1578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:35.979134 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:35.979138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:35.980689 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:35.981474 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:35.981481 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:35.981485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:35.982044 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:06:35.982201 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:35.983454 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0929 16:06:35.983752 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2040,"thread_start_us":1911,"threads_started":1}
I20260929 16:06:35.987463 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:35.949887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34633 (local address 127.20.121.129:58575)
0929 16:06:35.953267 (+ 3380us) negotiation.cc:107] Waiting for socket to connect
0929 16:06:35.953326 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 16:06:35.955945 (+ 2619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:06:35.957461 (+ 1516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:06:35.957478 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:06:35.958310 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:06:35.960248 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:35.960292 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:35.962290 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:35.962300 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:35.964914 (+ 2614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:35.964940 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:35.966761 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:35.966771 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:35.967578 (+ 807us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:35.968608 (+ 1030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:06:35.968647 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:06:35.972879 (+ 4232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:06:35.977875 (+ 4996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:06:35.977888 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:06:35.977912 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:06:35.978821 (+ 909us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:06:35.980929 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:06:35.980935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:06:35.980939 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:06:35.981303 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:06:35.982342 (+ 1039us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:06:35.982352 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:06:35.982918 (+ 566us) client_negotiation.cc:769] Sending connection context
0929 16:06:35.984910 (+ 1992us) client_negotiation.cc:240] Negotiation successful
0929 16:06:35.985662 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3051,"thread_start_us":1518,"threads_started":1}
I20260929 16:06:35.990363 32569 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34633
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:06:35.990796 32569 heartbeater.cc:464] Registering TS with master...
I20260929 16:06:35.991909 32569 heartbeater.cc:511] Master 127.20.121.190:34633 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:06:35.995090 32355 ts_manager.cc:194] Registered new tserver with Master: 97a48baaf8ad43109bfca58fc78c823d (127.20.121.129:44989)
I20260929 16:06:35.997447 32355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:58575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:06:36.039417 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:36513/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34633
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:06:36.541725 32576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:06:36.542143 32576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:06:36.542325 32576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:06:36.550808 32576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:06:36.550931 32576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:06:36.551119 32576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:06:36.561580 32576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.121.148:36513/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:06:36.563139 32576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:06:36.564503 32576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:06:36.581003 32588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:06:36.581121 32589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:06:36.581171 32591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:06:36.582106 32576 server_base.cc:1034] running on GCE node
I20260929 16:06:36.582854 32576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:06:36.583983 32576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:06:36.585328 32576 hybrid_clock.cc:648] HybridClock initialized: now 1790697996585276 us; error 43 us; skew 500 ppm
I20260929 16:06:36.585728 32576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:06:36 dist-test-slave-x07d krb5kdc[31879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697996, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:06:36.589481 32576 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:06:36.591467 32576 webserver.cc:460] Webserver started at http://127.20.121.130:43481/ using document root <none> and password file <none>
I20260929 16:06:36.592244 32576 fs_manager.cc:359] Metadata directory not provided
I20260929 16:06:36.592381 32576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:06:36.592767 32576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 16:06:36 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697996, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:06:36.611676 32576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "3f9ebfad717b46ad9e48bc58d1435c16"
format_stamp: "Formatted at 2026-09-29 16:06:36 on dist-test-slave-x07d"
server_key: "10a71f6cef3a846ffc05428215c6d52f"
server_key_iv: "a951f9895dff9248d7ca5480da53b949"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:36.613603 32576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "3f9ebfad717b46ad9e48bc58d1435c16"
format_stamp: "Formatted at 2026-09-29 16:06:36 on dist-test-slave-x07d"
server_key: "10a71f6cef3a846ffc05428215c6d52f"
server_key_iv: "a951f9895dff9248d7ca5480da53b949"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:36.623272 32576 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.004s
I20260929 16:06:36.652869 32598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:06:36.655087 32576 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:06:36.655418 32576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "3f9ebfad717b46ad9e48bc58d1435c16"
format_stamp: "Formatted at 2026-09-29 16:06:36 on dist-test-slave-x07d"
server_key: "10a71f6cef3a846ffc05428215c6d52f"
server_key_iv: "a951f9895dff9248d7ca5480da53b949"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:36.655834 32576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:06:36.694218 32576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:06:36.699769 32576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:06:36.700212 32576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:06:36.702040 32576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:06:36.704078 32576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:06:36.704236 32576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260929 16:06:36.704416 32576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:06:36.704509 32576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:06:36.751600 32576 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:41749
I20260929 16:06:36.751632 32710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:41749 every 8 connection(s)
I20260929 16:06:36.753575 32576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:06:36.758484 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 32576
I20260929 16:06:36.758683 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:06:36.759209 20966 external_mini_cluster.cc:1428] Setting key 3a8d3546c510ae45d62f68a83fecff05
Sep 29 16:06:36 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697996, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
I20260929 16:06:36.813613 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:36.756974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:48667 (local address 127.20.121.190:34633)
0929 16:06:36.772385 (+ 15411us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:36.772393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:36.789347 (+ 16954us) server_negotiation.cc:407] Connection header received
0929 16:06:36.792833 (+ 3486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:36.792843 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:36.792959 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:36.793105 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:06:36.795786 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:36.796958 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:36.799841 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:36.800787 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:36.800990 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:36.805218 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:06:36.805224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:06:36.805227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:06:36.805266 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:06:36.809013 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:36.810087 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:36.810096 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:36.810105 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:36.810273 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:36.810947 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:36.810956 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:36.810963 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:36.811436 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:06:36.811615 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:36.812713 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0929 16:06:36.813023 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15218,"thread_start_us":15443,"threads_started":1}
I20260929 16:06:36.819641 32714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:36.757809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34633 (local address 127.20.121.130:48667)
0929 16:06:36.788896 (+ 31087us) negotiation.cc:107] Waiting for socket to connect
0929 16:06:36.788968 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0929 16:06:36.792336 (+ 3368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:06:36.793688 (+ 1352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:06:36.793702 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:06:36.794505 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:06:36.795479 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:36.795511 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:36.797219 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:36.797225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:36.799543 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:36.799580 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:36.801143 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:36.801152 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:36.801537 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:36.802557 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:06:36.802590 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:06:36.804905 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:06:36.809217 (+ 4312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:06:36.809228 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:06:36.809250 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:06:36.809916 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:06:36.810447 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:06:36.810454 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:06:36.810459 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:06:36.810778 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:06:36.811616 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:06:36.811625 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:06:36.812120 (+ 495us) client_negotiation.cc:769] Sending connection context
0929 16:06:36.812552 (+ 432us) client_negotiation.cc:240] Negotiation successful
0929 16:06:36.813139 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30688,"thread_start_us":30472,"threads_started":1}
I20260929 16:06:36.821502 32711 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34633
I20260929 16:06:36.821873 32711 heartbeater.cc:464] Registering TS with master...
I20260929 16:06:36.822880 32711 heartbeater.cc:511] Master 127.20.121.190:34633 requested a full tablet report, sending...
I20260929 16:06:36.824911 32355 ts_manager.cc:194] Registered new tserver with Master: 3f9ebfad717b46ad9e48bc58d1435c16 (127.20.121.130:41749)
I20260929 16:06:36.826216 32355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:48667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:06:36.847627 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:36513/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34633
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:06:37.002429 32569 heartbeater.cc:503] Master 127.20.121.190:34633 was elected leader, sending a full tablet report...
W20260929 16:06:37.235620 32717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:06:37.235991 32717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:06:37.236181 32717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:06:37.244390 32717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:06:37.244555 32717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:06:37.244745 32717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:06:37.255170 32717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.121.148:36513/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:06:37.256650 32717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:06:37.258038 32717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:06:37.272464 32725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:06:37.272394 32723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:06:37.272732 32722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:06:37.273792 32717 server_base.cc:1034] running on GCE node
I20260929 16:06:37.274591 32717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:06:37.275874 32717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:06:37.277148 32717 hybrid_clock.cc:648] HybridClock initialized: now 1790697997277100 us; error 41 us; skew 500 ppm
I20260929 16:06:37.277503 32717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:06:37 dist-test-slave-x07d krb5kdc[31879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697997, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:06:37.281076 32717 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:06:37.282928 32717 webserver.cc:460] Webserver started at http://127.20.121.131:33367/ using document root <none> and password file <none>
I20260929 16:06:37.283651 32717 fs_manager.cc:359] Metadata directory not provided
I20260929 16:06:37.283816 32717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:06:37.284132 32717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 16:06:37 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697997, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:06:37.300990 32717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "b97af54e15174ce4a7e242442773820b"
format_stamp: "Formatted at 2026-09-29 16:06:37 on dist-test-slave-x07d"
server_key: "2ca47fbe2d85a8777a15cfd1db094b64"
server_key_iv: "9666b5d1469ffeb9d775fc758d52756f"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:37.301916 32717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "b97af54e15174ce4a7e242442773820b"
format_stamp: "Formatted at 2026-09-29 16:06:37 on dist-test-slave-x07d"
server_key: "2ca47fbe2d85a8777a15cfd1db094b64"
server_key_iv: "9666b5d1469ffeb9d775fc758d52756f"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:37.307570 32717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 16:06:37.325047 32732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:06:37.326752 32717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 16:06:37.326942 32717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "b97af54e15174ce4a7e242442773820b"
format_stamp: "Formatted at 2026-09-29 16:06:37 on dist-test-slave-x07d"
server_key: "2ca47fbe2d85a8777a15cfd1db094b64"
server_key_iv: "9666b5d1469ffeb9d775fc758d52756f"
server_key_version: "kuduclusterkey@0"
I20260929 16:06:37.327248 32717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:06:37.341246 32717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:06:37.345141 32717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:06:37.345453 32717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:06:37.346540 32717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:06:37.347951 32717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:06:37.348026 32717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:06:37.348184 32717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:06:37.348246 32717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:06:37.383615 32717 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:40931
I20260929 16:06:37.383649 376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:40931 every 8 connection(s)
I20260929 16:06:37.384886 32717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:06:37.386947 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 32717
I20260929 16:06:37.387120 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:06:37.387632 20966 external_mini_cluster.cc:1428] Setting key 068e559407af825d503fe5fbf123614e
Sep 29 16:06:37 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697997, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:06:37.407085 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.388405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:33939 (local address 127.20.121.190:34633)
0929 16:06:37.388933 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:37.388938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:37.390156 (+ 1218us) server_negotiation.cc:407] Connection header received
0929 16:06:37.392119 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:37.392123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:37.392190 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:37.392278 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:06:37.393720 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.394518 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.396110 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.396559 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.396656 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.400227 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:06:37.400233 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:06:37.400236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:06:37.400271 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:06:37.404047 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:37.404826 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:37.404831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:37.404833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:37.404959 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:37.405399 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:37.405402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:37.405404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:37.405675 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:06:37.405783 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:37.406637 (+ 854us) server_negotiation.cc:299] Negotiation successful
0929 16:06:37.406823 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":265,"threads_started":1}
I20260929 16:06:37.408119 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.388937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34633 (local address 127.20.121.131:33939)
0929 16:06:37.389968 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0929 16:06:37.390041 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0929 16:06:37.391867 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:06:37.392550 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:06:37.392559 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:06:37.392991 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:06:37.393566 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.393583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.394666 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.394670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.395960 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.395981 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.396770 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.396775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.396982 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.397780 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:06:37.397817 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:06:37.399980 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:06:37.404196 (+ 4216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:06:37.404204 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:06:37.404220 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:06:37.404710 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:06:37.405095 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:06:37.405099 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:06:37.405101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:06:37.405305 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:06:37.405794 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:06:37.405801 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:06:37.406260 (+ 459us) client_negotiation.cc:769] Sending connection context
0929 16:06:37.406532 (+ 272us) client_negotiation.cc:240] Negotiation successful
0929 16:06:37.406947 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":349,"threads_started":1}
I20260929 16:06:37.409644 377 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34633
I20260929 16:06:37.409997 377 heartbeater.cc:464] Registering TS with master...
I20260929 16:06:37.410784 377 heartbeater.cc:511] Master 127.20.121.190:34633 requested a full tablet report, sending...
I20260929 16:06:37.412297 32355 ts_manager.cc:194] Registered new tserver with Master: b97af54e15174ce4a7e242442773820b (127.20.121.131:40931)
I20260929 16:06:37.413157 32355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:33939
I20260929 16:06:37.416338 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:06:37.436455 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.421334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57830 (local address 127.20.121.190:34633)
0929 16:06:37.421474 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:37.421479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:37.422209 (+ 730us) server_negotiation.cc:407] Connection header received
0929 16:06:37.422277 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:37.422281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:37.422359 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:37.422449 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:06:37.424017 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.424773 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.426650 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.427157 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.427281 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.430196 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:06:37.430201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:06:37.430205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:06:37.430245 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:06:37.433518 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:37.434504 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:37.434509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:37.434511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:37.434649 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:37.435111 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:37.435116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:37.435119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:37.435425 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:06:37.435644 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:37.436019 (+ 375us) server_negotiation.cc:299] Negotiation successful
0929 16:06:37.436209 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 16:06:37.441424 32355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:06:37.449577 32355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:06:37.476058 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.465203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.20.121.131:40931)
0929 16:06:37.467096 (+ 1893us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:37.467106 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:37.467141 (+ 35us) server_negotiation.cc:407] Connection header received
0929 16:06:37.467248 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:37.467256 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:37.467542 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:37.468565 (+ 1023us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:06:37.468828 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.470267 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.473601 (+ 3334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.474828 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.475053 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.475254 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:37.475526 (+ 272us) server_negotiation.cc:299] Negotiation successful
0929 16:06:37.475698 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1724,"thread_start_us":2769,"threads_started":1}
I20260929 16:06:37.476953 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.467314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:41749 (local address 127.0.0.1:56548)
0929 16:06:37.467806 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 16:06:37.467828 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:06:37.467941 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:06:37.469264 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:06:37.469268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:06:37.469290 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:06:37.469809 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.469819 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.471785 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.471791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.474038 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.474053 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.476068 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.476076 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.476269 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.476300 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 16:06:37.476397 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 16:06:37.476537 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":1,"thread_start_us":1066,"threads_started":1}
I20260929 16:06:37.476985 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.464751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40931 (local address 127.0.0.1:53112)
0929 16:06:37.466273 (+ 1522us) negotiation.cc:107] Waiting for socket to connect
0929 16:06:37.466327 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 16:06:37.466519 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:06:37.467802 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:06:37.467809 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:06:37.467842 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:06:37.468352 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.468367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.470715 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.470720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.473387 (+ 2667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.473400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.475053 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.475059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.475243 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.475297 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 16:06:37.475481 (+ 184us) client_negotiation.cc:240] Negotiation successful
0929 16:06:37.476462 (+ 981us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1178,"mutex_wait_us":761,"spinlock_wait_cycles":1792,"thread_start_us":3414,"threads_started":1}
I20260929 16:06:37.477975 395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.467786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56548 (local address 127.20.121.130:41749)
0929 16:06:37.468626 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:37.468636 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:37.468679 (+ 43us) server_negotiation.cc:407] Connection header received
0929 16:06:37.468787 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:37.468795 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:37.469049 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:37.469255 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:06:37.470063 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.471612 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.474224 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.475826 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.476024 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.476199 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:37.477539 (+ 1340us) server_negotiation.cc:299] Negotiation successful
0929 16:06:37.477674 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":792,"threads_started":1}
I20260929 16:06:37.480330 393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.464797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:44989 (local address 127.0.0.1:40216)
0929 16:06:37.467197 (+ 2400us) negotiation.cc:107] Waiting for socket to connect
0929 16:06:37.467224 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:06:37.467353 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:06:37.468326 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:06:37.468333 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:06:37.468434 (+ 101us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:06:37.471195 (+ 2761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.471210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.473327 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.473332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.477234 (+ 3902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.477251 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.479427 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.479434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.479639 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.479679 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 16:06:37.479764 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 16:06:37.479929 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2287,"mutex_wait_us":50,"thread_start_us":1388,"threads_started":1}
I20260929 16:06:37.481714 312 tablet_service.cc:1467] Processing CreateTablet for tablet b31d805761184ab88adc5556b68a134d (DEFAULT_TABLE table=test-table [id=cb3d87e501f444fba3135a731e175ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:06:37.482404 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.465195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.20.121.129:44989)
0929 16:06:37.466901 (+ 1706us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:37.466912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:37.467481 (+ 569us) server_negotiation.cc:407] Connection header received
0929 16:06:37.467607 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:37.467615 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:37.467859 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:37.468055 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:06:37.471383 (+ 3328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.473150 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.477489 (+ 4339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.479194 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.481448 (+ 2254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.481695 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:37.481795 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 16:06:37.481997 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1518,"thread_start_us":1356,"threads_started":1}
I20260929 16:06:37.482636 32646 tablet_service.cc:1467] Processing CreateTablet for tablet b31d805761184ab88adc5556b68a134d (DEFAULT_TABLE table=test-table [id=cb3d87e501f444fba3135a731e175ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:06:37.483387 312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b31d805761184ab88adc5556b68a134d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:06:37.483953 32646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b31d805761184ab88adc5556b68a134d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:06:37.486645 32503 tablet_service.cc:1467] Processing CreateTablet for tablet b31d805761184ab88adc5556b68a134d (DEFAULT_TABLE table=test-table [id=cb3d87e501f444fba3135a731e175ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:06:37.488564 32503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b31d805761184ab88adc5556b68a134d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:06:37.497746 396 tablet_bootstrap.cc:492] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: Bootstrap starting.
I20260929 16:06:37.502907 397 tablet_bootstrap.cc:492] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: Bootstrap starting.
I20260929 16:06:37.503304 396 tablet_bootstrap.cc:654] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: Neither blocks nor log segments found. Creating new log.
I20260929 16:06:37.505585 396 log.cc:824] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: Log is configured to *not* fsync() on all Append() calls
I20260929 16:06:37.507592 397 tablet_bootstrap.cc:654] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: Neither blocks nor log segments found. Creating new log.
I20260929 16:06:37.507229 399 tablet_bootstrap.cc:492] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: Bootstrap starting.
I20260929 16:06:37.508908 397 log.cc:824] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: Log is configured to *not* fsync() on all Append() calls
I20260929 16:06:37.509261 396 tablet_bootstrap.cc:492] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: No bootstrap required, opened a new log
I20260929 16:06:37.509600 396 ts_tablet_manager.cc:1397] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20260929 16:06:37.512512 399 tablet_bootstrap.cc:654] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: Neither blocks nor log segments found. Creating new log.
I20260929 16:06:37.512955 397 tablet_bootstrap.cc:492] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: No bootstrap required, opened a new log
I20260929 16:06:37.513274 397 ts_tablet_manager.cc:1397] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260929 16:06:37.513814 399 log.cc:824] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: Log is configured to *not* fsync() on all Append() calls
I20260929 16:06:37.520493 399 tablet_bootstrap.cc:492] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: No bootstrap required, opened a new log
I20260929 16:06:37.520814 399 ts_tablet_manager.cc:1397] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260929 16:06:37.520736 396 raft_consensus.cc:348] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 } } peers { permanent_uuid: "97a48baaf8ad43109bfca58fc78c823d" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44989 } } peers { permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 } }
I20260929 16:06:37.521205 396 raft_consensus.cc:374] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:06:37.521309 396 raft_consensus.cc:729] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f9ebfad717b46ad9e48bc58d1435c16, State: Initialized, Role: FOLLOWER
I20260929 16:06:37.521970 396 consensus_queue.cc:260] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 } } peers { permanent_uuid: "97a48baaf8ad43109bfca58fc78c823d" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44989 } } peers { permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 } }
I20260929 16:06:37.523188 32711 heartbeater.cc:503] Master 127.20.121.190:34633 was elected leader, sending a full tablet report...
I20260929 16:06:37.524480 397 raft_consensus.cc:348] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 } } peers { permanent_uuid: "97a48baaf8ad43109bfca58fc78c823d" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44989 } } peers { permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 } }
I20260929 16:06:37.524909 397 raft_consensus.cc:374] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:06:37.525029 397 raft_consensus.cc:729] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b97af54e15174ce4a7e242442773820b, State: Initialized, Role: FOLLOWER
I20260929 16:06:37.525403 396 ts_tablet_manager.cc:1428] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260929 16:06:37.525743 397 consensus_queue.cc:260] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 } } peers { permanent_uuid: "97a48baaf8ad43109bfca58fc78c823d" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44989 } } peers { permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 } }
I20260929 16:06:37.529085 377 heartbeater.cc:503] Master 127.20.121.190:34633 was elected leader, sending a full tablet report...
I20260929 16:06:37.530225 397 ts_tablet_manager.cc:1428] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
I20260929 16:06:37.531332 399 raft_consensus.cc:348] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 } } peers { permanent_uuid: "97a48baaf8ad43109bfca58fc78c823d" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44989 } } peers { permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 } }
I20260929 16:06:37.531889 399 raft_consensus.cc:374] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:06:37.532015 399 raft_consensus.cc:729] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a48baaf8ad43109bfca58fc78c823d, State: Initialized, Role: FOLLOWER
I20260929 16:06:37.532713 399 consensus_queue.cc:260] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 } } peers { permanent_uuid: "97a48baaf8ad43109bfca58fc78c823d" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44989 } } peers { permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 } }
I20260929 16:06:37.534592 399 ts_tablet_manager.cc:1428] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
W20260929 16:06:37.540863 32712 tablet.cc:2369] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:06:37.630839 404 raft_consensus.cc:482] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:06:37.631172 404 raft_consensus.cc:504] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 } } peers { permanent_uuid: "97a48baaf8ad43109bfca58fc78c823d" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44989 } } peers { permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 } }
I20260929 16:06:37.633415 404 leader_election.cc:288] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b97af54e15174ce4a7e242442773820b (127.20.121.131:40931), 3f9ebfad717b46ad9e48bc58d1435c16 (127.20.121.130:41749)
W20260929 16:06:37.637148 378 tablet.cc:2369] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:06:37.641654 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.633989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:53579 (local address 127.20.121.131:40931)
0929 16:06:37.634245 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:37.634253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:37.634794 (+ 541us) server_negotiation.cc:407] Connection header received
0929 16:06:37.634947 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:37.634954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:37.635034 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:37.635140 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:06:37.636042 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.636856 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.639662 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.640570 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.640744 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.640836 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:37.641229 (+ 393us) server_negotiation.cc:299] Negotiation successful
0929 16:06:37.641386 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260929 16:06:37.642059 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.633747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40931 (local address 127.20.121.129:53579)
0929 16:06:37.634680 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0929 16:06:37.634695 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 16:06:37.634801 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:06:37.635339 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:06:37.635345 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:06:37.635385 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:06:37.635883 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.635901 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.637049 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.637054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.639448 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.639466 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.640814 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.640821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.641030 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.641079 (+ 49us) client_negotiation.cc:769] Sending connection context
0929 16:06:37.641186 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 16:06:37.641350 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":269,"spinlock_wait_cycles":2432,"thread_start_us":458,"threads_started":1}
I20260929 16:06:37.643221 405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.633734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:41749 (local address 127.20.121.129:60043)
0929 16:06:37.634553 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0929 16:06:37.634577 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:06:37.634797 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:06:37.635190 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:06:37.635195 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:06:37.635223 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:06:37.635691 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.635711 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.637724 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.637730 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.640252 (+ 2522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:06:37.640271 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.642275 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.642282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:06:37.642473 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.642510 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 16:06:37.642640 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 16:06:37.642810 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":18,"thread_start_us":460,"threads_started":1}
I20260929 16:06:37.643306 332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31d805761184ab88adc5556b68a134d" candidate_uuid: "97a48baaf8ad43109bfca58fc78c823d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b97af54e15174ce4a7e242442773820b" is_pre_election: true
I20260929 16:06:37.643983 332 raft_consensus.cc:2454] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97a48baaf8ad43109bfca58fc78c823d in term 0.
I20260929 16:06:37.644743 395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.634075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:60043 (local address 127.20.121.130:41749)
0929 16:06:37.634312 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:37.634318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:37.634672 (+ 354us) server_negotiation.cc:407] Connection header received
0929 16:06:37.634996 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:37.635000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:37.635056 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:37.635158 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:06:37.635887 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.637541 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.640477 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.641701 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.642385 (+ 684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.642463 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:37.642667 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 16:06:37.644539 (+ 1872us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 16:06:37.644760 32456 leader_election.cc:302] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97a48baaf8ad43109bfca58fc78c823d, b97af54e15174ce4a7e242442773820b; no voters:
I20260929 16:06:37.645229 404 raft_consensus.cc:2781] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:06:37.645462 404 raft_consensus.cc:482] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:06:37.645597 404 raft_consensus.cc:3037] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:06:37.645855 32666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31d805761184ab88adc5556b68a134d" candidate_uuid: "97a48baaf8ad43109bfca58fc78c823d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" is_pre_election: true
I20260929 16:06:37.646476 32666 raft_consensus.cc:2454] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97a48baaf8ad43109bfca58fc78c823d in term 0.
I20260929 16:06:37.648698 404 raft_consensus.cc:504] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 } } peers { permanent_uuid: "97a48baaf8ad43109bfca58fc78c823d" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44989 } } peers { permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 } }
I20260929 16:06:37.649473 404 leader_election.cc:288] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [CANDIDATE]: Term 1 election: Requested vote from peers b97af54e15174ce4a7e242442773820b (127.20.121.131:40931), 3f9ebfad717b46ad9e48bc58d1435c16 (127.20.121.130:41749)
I20260929 16:06:37.650146 332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31d805761184ab88adc5556b68a134d" candidate_uuid: "97a48baaf8ad43109bfca58fc78c823d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b97af54e15174ce4a7e242442773820b"
I20260929 16:06:37.650282 32666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31d805761184ab88adc5556b68a134d" candidate_uuid: "97a48baaf8ad43109bfca58fc78c823d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f9ebfad717b46ad9e48bc58d1435c16"
I20260929 16:06:37.650401 332 raft_consensus.cc:3037] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:06:37.650535 32666 raft_consensus.cc:3037] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:06:37.652827 332 raft_consensus.cc:2454] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97a48baaf8ad43109bfca58fc78c823d in term 1.
I20260929 16:06:37.653472 32456 leader_election.cc:302] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97a48baaf8ad43109bfca58fc78c823d, b97af54e15174ce4a7e242442773820b; no voters:
I20260929 16:06:37.653903 404 raft_consensus.cc:2781] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:06:37.654098 32666 raft_consensus.cc:2454] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97a48baaf8ad43109bfca58fc78c823d in term 1.
I20260929 16:06:37.654556 404 raft_consensus.cc:686] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 1 LEADER]: Becoming Leader. State: Replica: 97a48baaf8ad43109bfca58fc78c823d, State: Running, Role: LEADER
I20260929 16:06:37.655153 404 consensus_queue.cc:237] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 } } peers { permanent_uuid: "97a48baaf8ad43109bfca58fc78c823d" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44989 } } peers { permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 } }
I20260929 16:06:37.660235 32354 catalog_manager.cc:5495] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d reported cstate change: term changed from 0 to 1, leader changed from <none> to 97a48baaf8ad43109bfca58fc78c823d (127.20.121.129). New cstate: current_term: 1 leader_uuid: "97a48baaf8ad43109bfca58fc78c823d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97a48baaf8ad43109bfca58fc78c823d" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 44989 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 } health_report { overall_health: UNKNOWN } } }
I20260929 16:06:37.685070 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.677859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.20.121.129:44989)
0929 16:06:37.678150 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:37.678155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:37.678176 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:06:37.678378 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:37.678391 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:37.678482 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:37.678739 (+ 257us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:06:37.679173 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.680520 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.682653 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.683166 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.683322 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.683910 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:06:37.684459 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:06:37.684636 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:37.684699 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 16:06:37.684787 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 16:06:37.694543 332 raft_consensus.cc:1261] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b [term 1 FOLLOWER]: Refusing update from remote peer 97a48baaf8ad43109bfca58fc78c823d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:06:37.694631 32666 raft_consensus.cc:1261] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16 [term 1 FOLLOWER]: Refusing update from remote peer 97a48baaf8ad43109bfca58fc78c823d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:06:37.695652 404 consensus_queue.cc:1035] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f9ebfad717b46ad9e48bc58d1435c16" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:06:37.695967 407 consensus_queue.cc:1035] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b97af54e15174ce4a7e242442773820b" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 16:06:37.700559 32570 tablet.cc:2369] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:06:37.710052 410 mvcc.cc:205] Tried to move back new op lower bound from 7334698998545383424 to 7334698998400053248. Current Snapshot: MvccSnapshot[applied={T|T < 7334698998545383424}]
I20260929 16:06:37.725824 32354 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:06:37.726190 32354 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:06:37.730355 32354 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e3d25b91f284998bcc6e3783b61ee2f: Sending DeleteTablet for 3 replicas of tablet b31d805761184ab88adc5556b68a134d
I20260929 16:06:37.731899 32503 tablet_service.cc:1514] Processing DeleteTablet for tablet b31d805761184ab88adc5556b68a134d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:06:37 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:40216
I20260929 16:06:37.731855 312 tablet_service.cc:1514] Processing DeleteTablet for tablet b31d805761184ab88adc5556b68a134d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:06:37 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:53112
I20260929 16:06:37.733026 32646 tablet_service.cc:1514] Processing DeleteTablet for tablet b31d805761184ab88adc5556b68a134d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:06:37 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:56548
I20260929 16:06:37.735188 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 32435
I20260929 16:06:37.744153 420 tablet_replica.cc:330] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: stopping tablet replica
I20260929 16:06:37.744887 420 raft_consensus.cc:2229] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:06:37.745920 420 raft_consensus.cc:2258] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:06:37.748533 422 tablet_replica.cc:330] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: stopping tablet replica
I20260929 16:06:37.748571 421 tablet_replica.cc:330] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: stopping tablet replica
I20260929 16:06:37.749106 421 raft_consensus.cc:2229] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:06:37.749159 422 raft_consensus.cc:2229] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:06:37.749718 421 raft_consensus.cc:2258] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:06:37.749776 422 raft_consensus.cc:2258] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:06:37.752521 420 ts_tablet_manager.cc:1905] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 16:06:37 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:06:37.762300 420 ts_tablet_manager.cc:1918] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:06:37.762605 420 log.cc:1196] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/b31d805761184ab88adc5556b68a134d
I20260929 16:06:37.763216 421 ts_tablet_manager.cc:1905] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:06:37.763419 420 ts_tablet_manager.cc:1939] T b31d805761184ab88adc5556b68a134d P 97a48baaf8ad43109bfca58fc78c823d: Deleting consensus metadata
I20260929 16:06:37.772281 422 ts_tablet_manager.cc:1905] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:06:37.772595 32342 catalog_manager.cc:4841] TS 97a48baaf8ad43109bfca58fc78c823d (127.20.121.129:44989): tablet b31d805761184ab88adc5556b68a134d (table test-table [id=cb3d87e501f444fba3135a731e175ea7]) successfully deleted
I20260929 16:06:37.773023 421 ts_tablet_manager.cc:1918] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:06:37.773298 421 log.cc:1196] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/b31d805761184ab88adc5556b68a134d
I20260929 16:06:37.773967 421 ts_tablet_manager.cc:1939] T b31d805761184ab88adc5556b68a134d P b97af54e15174ce4a7e242442773820b: Deleting consensus metadata
I20260929 16:06:37.774899 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.736740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40234 (local address 127.20.121.129:44989)
0929 16:06:37.737948 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:37.737959 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:37.748952 (+ 10993us) server_negotiation.cc:407] Connection header received
0929 16:06:37.749520 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:37.749526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:37.749623 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:37.749719 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:06:37.752012 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.753224 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.755579 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.756377 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.756512 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.763118 (+ 6606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:06:37.763135 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:06:37.763162 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:06:37.763228 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:06:37.768280 (+ 5052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:37.770282 (+ 2002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:37.770295 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:37.770298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:37.770644 (+ 346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:37.772418 (+ 1774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:37.772424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:37.772428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:37.772981 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:06:37.773341 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:37.774138 (+ 797us) server_negotiation.cc:299] Negotiation successful
0929 16:06:37.774448 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":989}
I20260929 16:06:37.776520 32339 catalog_manager.cc:4841] TS b97af54e15174ce4a7e242442773820b (127.20.121.131:40931): tablet b31d805761184ab88adc5556b68a134d (table test-table [id=cb3d87e501f444fba3135a731e175ea7]) successfully deleted
I20260929 16:06:37.777127 32563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:06:37.781493 422 ts_tablet_manager.cc:1918] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:06:37.781733 422 log.cc:1196] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/b31d805761184ab88adc5556b68a134d
I20260929 16:06:37.782362 422 ts_tablet_manager.cc:1939] T b31d805761184ab88adc5556b68a134d P 3f9ebfad717b46ad9e48bc58d1435c16: Deleting consensus metadata
I20260929 16:06:37.784077 32342 catalog_manager.cc:4841] TS 3f9ebfad717b46ad9e48bc58d1435c16 (127.20.121.130:41749): tablet b31d805761184ab88adc5556b68a134d (table test-table [id=cb3d87e501f444fba3135a731e175ea7]) successfully deleted
I20260929 16:06:37.884433 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 32435
I20260929 16:06:37.904155 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 32576
Sep 29 16:06:37 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:06:37.920894 395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:37.905112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56558 (local address 127.20.121.130:41749)
0929 16:06:37.905338 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:37.905343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:37.905363 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:06:37.905423 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:37.905428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:37.905503 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:37.905606 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:06:37.907012 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.907807 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.909326 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:37.909738 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:37.909848 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:37.914562 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:06:37.914579 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:06:37.914595 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:06:37.914649 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:06:37.917757 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:37.918732 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:37.918740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:37.918742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:37.918953 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:37.919322 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:37.919325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:37.919327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:37.919723 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:06:37.920003 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:37.920302 (+ 299us) server_negotiation.cc:299] Negotiation successful
0929 16:06:37.920489 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260929 16:06:37.921710 32706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:06:38.013137 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 32576
I20260929 16:06:38.030318 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 32717
Sep 29 16:06:38 dist-test-slave-x07d krb5kdc[31879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790697992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:06:38.047101 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:06:38.031181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53120 (local address 127.20.121.131:40931)
0929 16:06:38.031385 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 16:06:38.031392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:06:38.031497 (+ 105us) server_negotiation.cc:407] Connection header received
0929 16:06:38.031707 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:06:38.031713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:06:38.031887 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:06:38.032012 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:06:38.033454 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:38.034280 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:38.035582 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:06:38.035975 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:06:38.036111 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:06:38.041002 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:06:38.041018 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:06:38.041036 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:06:38.041074 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:06:38.044106 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:38.044882 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:38.044890 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:38.044892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:38.045033 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:06:38.045427 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:06:38.045432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:06:38.045435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:06:38.045829 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:06:38.046166 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0929 16:06:38.046522 (+ 356us) server_negotiation.cc:299] Negotiation successful
0929 16:06:38.046748 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:06:38.047773 372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:06:38.140992 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 32717
I20260929 16:06:38.161859 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 32323
I20260929 16:06:38.163211 32385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:06:38.277626 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 32323
I20260929 16:06:38.304558 20966 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 16:06:38.641554 20966 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 16:06:38.641846 20966 mini_ranger.cc:66] Stopping Ranger...
I20260929 16:06:38.988911 20966 mini_ranger.cc:68] Stopped Ranger
2026-09-29 16:06:38.989 UTC [31909] LOG: received smart shutdown request
2026-09-29 16:06:38.993 UTC [31909] LOG: background worker "logical replication launcher" (PID 31918) exited with exit code 1
2026-09-29 16:06:38.994 UTC [31913] LOG: shutting down
2026-09-29 16:06:39.245 UTC [31909] LOG: database system is shut down
2026-09-29T16:06:39Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53780 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:06:39 dist-test-slave-x07d krb5kdc[436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:06:39 dist-test-slave-x07d krb5kdc[436](info): set up 2 sockets
Sep 29 16:06:39 dist-test-slave-x07d krb5kdc[436](info): commencing operation
krb5kdc: starting...
W20260929 16:06:40.502719 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.002s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:06:40 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:06:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:06:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.121.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.121.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/rangeradmin_127.20.121.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/rangeradmin_127.20.121.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.121.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.121.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/rangerlookup_127.20.121.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/rangerlookup_127.20.121.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.121.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.121.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/HTTP_127.20.121.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/HTTP_127.20.121.148@KRBTEST.COM.keytab.
I20260929 16:06:40.855044 20966 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskDT3jwB/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/postgres -l logfile start
2026-09-29 16:06:44.215 UTC [466] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 16:06:44.215 UTC [466] LOG: listening on IPv4 address "127.20.121.148", port 33919
2026-09-29 16:06:44.220 UTC [466] LOG: listening on Unix socket "/tmp/.s.PGSQL.33919"
2026-09-29 16:06:44.245 UTC [469] LOG: database system was shut down at 2026-09-29 16:06:41 UTC
2026-09-29 16:06:44.250 UTC [466] LOG: database system is ready to accept connections
I20260929 16:06:45.294790 20966 mini_postgres.cc:96] Postgres bound to 33919
2026-09-29 16:06:45.314 UTC [477] FATAL: database "slave" does not exist
127.20.121.148:33919 - accepting connections
I20260929 16:06:45.314934 20966 mini_ranger.cc:161] Starting Ranger...
I20260929 16:06:45.346602 20966 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 16:06:46.237396 20966 mini_ranger.cc:87] Created ranger Postgres database
I20260929 16:06:46.237532 20966 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-admin
2026-09-29 16:06:46,552 [I] DB FLAVOR :POSTGRES
2026-09-29 16:06:46,552 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 16:06:46,552 [I] Checking connection..
2026-09-29 16:06:46,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 16:06:46,813 [I] Checking connection passed.
2026-09-29 16:06:46,813 [I] --------- Verifying version history table ---------
2026-09-29 16:06:46,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 16:06:47,102 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 16:06:47,103 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 16:06:47,380 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 16:06:47,380 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 16:06:47,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 16:06:47.640 UTC [541] WARNING: there is no transaction in progress
2026-09-29 16:06:47,646 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 16:06:47,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 16:06:47,923 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 16:06:47,923 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 16:06:47,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 16:06:48,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 16:06:48,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x07d.c.gcp-upstream.internal','N') ;"
2026-09-29 16:06:48,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 16:06:48,970 [I] Table x_portal_user does not exist in database ranger
2026-09-29 16:06:48,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 16:06:49,253 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 16:06:49,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 16:06:49,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 16:06:49,752 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 16:06:49,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 16:06:50.886 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:50.905 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:50.925 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:50.944 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:50.962 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:50.983 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51.050 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51.058 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51.075 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51.087 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51.099 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51.115 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51.126 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51.134 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51.143 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51.151 UTC [649] WARNING: there is no transaction in progress
2026-09-29 16:06:51,992 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 16:06:51,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 16:06:52,253 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 16:06:52,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 16:06:52,508 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 16:06:52,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 16:06:52,788 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 16:06:52,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 16:06:53,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 16:06:53,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x07d.c.gcp-upstream.internal';"
2026-09-29 16:06:53,553 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 16:06:53,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 16:06:53,822 [I] DB_PATCHES have already been applied
I20260929 16:06:53.828904 20966 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-admin:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskDT3jwB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 16:06:53.829010 20966 mini_ranger.cc:193] Using host: 127.20.121.148
I20260929 16:06:53.845039 20966 mini_ranger.cc:227] Ranger admin URL: http://127.20.121.148:53677
Sep 29, 2026 4:06:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 4:06:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 4:06:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 4:06:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 4:06:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 4:06:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 4:06:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 16:06:55.138620 20966 mini_ranger.cc:160] Time spent starting Ranger: real 9.824s user 0.001s sys 0.050s
Sep 29, 2026 4:07:16 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 4:07:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 4:07:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 4:07:17 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 4:07:18 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 16:07:18.766544 20966 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.121.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.121.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/rangerkms_127.20.121.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.121.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/rangerkms_127.20.121.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.121.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.121.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.121.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/HTTP_127.20.121.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.121.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/HTTP_127.20.121.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 16:07:18 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698038, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 16:07:18.965323 20966 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 16:07:18.989887 20966 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 16:07:20.289121 20966 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 16:07:20.289215 20966 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-kms
I20260929 16:07:20.289252 20966 mini_ranger_kms.cc:227] Using postgres at 127.20.121.148:33919
2026-09-29 16:07:20,678 [I] DB FLAVOR :POSTGRES
2026-09-29 16:07:20,679 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 16:07:20,679 [I] Checking connection
2026-09-29 16:07:20,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 16:07:20,945 [I] connection success
2026-09-29 16:07:20,945 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 16:07:20,945 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 16:07:20,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 16:07:21,191 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 16:07:21,192 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 16:07:21,192 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 16:07:21,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.121.148:33919/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 16:07:21,459 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 16:07:22.021908 20966 mini_ranger_kms.cc:310] Created kms service
I20260929 16:07:22.203399 20966 mini_ranger_kms.cc:326] Created kudu user
I20260929 16:07:22.250325 20966 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 16:07:22 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698038, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:22.688387 20966 mini_ranger_kms.cc:384] Added ranger policy
I20260929 16:07:22.688726 20966 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-kms:/tmp/dist-test-taskDT3jwB/build/asan/bin/postgresql.jar:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskDT3jwB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 16:07:22.688813 20966 mini_ranger_kms.cc:240] Using host: 127.20.121.148
I20260929 16:07:22.704121 20966 mini_ranger_kms.cc:276] Ranger KMS PID: 830
I20260929 16:07:22.704269 20966 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.121.148:47425
Sep 29, 2026 4:07:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 4:07:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 4:07:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 4:07:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 4:07:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 4:07:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 29 16:07:25 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790698045, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.121.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:25.053681 20966 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.088s user 0.082s sys 0.320s
I20260929 16:07:25.053939 20966 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 16:07:25.054148 20966 mini_ranger_kms.cc:401] 127.20.121.148:47425/kms/v1/keys
Sep 29 16:07:25 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698045, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.121.148@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
Sep 29 16:07:25 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698045, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.121.148@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
Sep 29 16:07:28 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:07:28.509711 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37739
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:37739
--ranger_config_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:47425/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:07:28.865218 880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:28.865619 880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:28.865777 880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:28.873457 880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:07:28.873667 880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:28.873756 880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:28.873862 880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:07:28.873950 880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:07:28.884864 880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.121.148:47425/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:37739
--ranger_config_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:28.886389 880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:28.887780 880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:28.901525 885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:28.901489 888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:28.901438 886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:28.902637 880 server_base.cc:1034] running on GCE node
I20260929 16:07:28.903636 880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:28.905244 880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:28.906507 880 hybrid_clock.cc:648] HybridClock initialized: now 1790698048906463 us; error 31 us; skew 500 ppm
I20260929 16:07:28.906910 880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:28 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:28.910895 880 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:07:28.912541 880 webserver.cc:460] Webserver started at http://127.20.121.190:33155/ using document root <none> and password file <none>
I20260929 16:07:28.913254 880 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:28.913404 880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:28.913727 880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 16:07:28 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698048, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:28.974995 880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "b2b71ab6b54748df996139a2f634f4b0"
format_stamp: "Formatted at 2026-09-29 16:07:28 on dist-test-slave-x07d"
server_key: "dd22dabf285e882c254125567b9bd6fc"
server_key_iv: "619adcce1d724af1339a0f7bf183f9bf"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:28.975999 880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "b2b71ab6b54748df996139a2f634f4b0"
format_stamp: "Formatted at 2026-09-29 16:07:28 on dist-test-slave-x07d"
server_key: "dd22dabf285e882c254125567b9bd6fc"
server_key_iv: "619adcce1d724af1339a0f7bf183f9bf"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:28.982287 880 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260929 16:07:29.000366 895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:29.002050 880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:07:29.002281 880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "b2b71ab6b54748df996139a2f634f4b0"
format_stamp: "Formatted at 2026-09-29 16:07:28 on dist-test-slave-x07d"
server_key: "dd22dabf285e882c254125567b9bd6fc"
server_key_iv: "619adcce1d724af1339a0f7bf183f9bf"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:29.002522 880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:29.019805 880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:29.024358 880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:29.024698 880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:29.045958 880 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:37739
I20260929 16:07:29.045965 946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:37739 every 8 connection(s)
I20260929 16:07:29.048327 880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:07:29.048681 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 880
I20260929 16:07:29.048915 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:07:29.050455 20966 external_mini_cluster.cc:1428] Setting key f708f0950274a2060f6b0f7c51b1fcd6
I20260929 16:07:29.054113 947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:29.071476 947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: Bootstrap starting.
Sep 29 16:07:29 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:29.075544 947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:29.076944 947 log.cc:824] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:29.081489 947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: No bootstrap required, opened a new log
I20260929 16:07:29.089040 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:29.055562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.20.121.190:37739)
0929 16:07:29.058681 (+ 3119us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:29.058694 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:29.058739 (+ 45us) server_negotiation.cc:407] Connection header received
0929 16:07:29.061931 (+ 3192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:29.061962 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:29.062667 (+ 705us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:29.064942 (+ 2275us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:29.065549 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:29.067660 (+ 2111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:29.069734 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:29.070886 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:29.071790 (+ 904us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:29.076426 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:29.076439 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:29.076462 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:29.076520 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:29.083566 (+ 7046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:29.084598 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:29.084606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:29.084615 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:29.084820 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:29.085303 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:29.085307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:29.085309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:29.085984 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:29.086434 (+ 450us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:29.086909 (+ 475us) server_negotiation.cc:299] Negotiation successful
0929 16:07:29.087492 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":2814,"thread_start_us":2415,"threads_started":1}
I20260929 16:07:29.093626 947 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } }
I20260929 16:07:29.094218 947 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:29.094355 947 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b71ab6b54748df996139a2f634f4b0, State: Initialized, Role: FOLLOWER
I20260929 16:07:29.095165 947 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } }
I20260929 16:07:29.095484 947 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:07:29.095690 947 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:07:29.095924 947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:29.099359 947 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } }
I20260929 16:07:29.100111 947 leader_election.cc:302] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2b71ab6b54748df996139a2f634f4b0; no voters:
I20260929 16:07:29.101022 947 leader_election.cc:288] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:07:29.101164 952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:07:29.101863 952 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 1 LEADER]: Becoming Leader. State: Replica: b2b71ab6b54748df996139a2f634f4b0, State: Running, Role: LEADER
I20260929 16:07:29.102559 952 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } }
I20260929 16:07:29.104547 947 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:07:29.105540 954 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2b71ab6b54748df996139a2f634f4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } } }
I20260929 16:07:29.106053 954 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:29.106653 953 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2b71ab6b54748df996139a2f634f4b0. Latest consensus state: current_term: 1 leader_uuid: "b2b71ab6b54748df996139a2f634f4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } } }
I20260929 16:07:29.106949 953 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:29.110224 957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:07:29.115511 957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:07:29.137775 957 catalog_manager.cc:1347] Generated new cluster ID: b546e91dcdfe4018945c750a0465f188
I20260929 16:07:29.137949 957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:07:29.164978 957 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:07:29.167693 957 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:07:29.172571 947 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 16:07:29.194453 957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: Generated new TSK 0
I20260929 16:07:29.196035 957 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:07:30.086570 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:47425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37739
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:07:30 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790698050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 16:07:30.565073 992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:30.565572 992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:30.565779 992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:30.578171 992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:30.578332 992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:30.578577 992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:07:30.594873 992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.121.148:47425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:30.596655 992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:30.598414 992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:30.613042 1002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:30.613210 1003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:30.613895 1005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:30.614113 992 server_base.cc:1034] running on GCE node
I20260929 16:07:30.614893 992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:30.616060 992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:30.617316 992 hybrid_clock.cc:648] HybridClock initialized: now 1790698050617246 us; error 75 us; skew 500 ppm
I20260929 16:07:30.617672 992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:30 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:30.621588 992 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:07:30.641101 992 webserver.cc:460] Webserver started at http://127.20.121.129:46745/ using document root <none> and password file <none>
I20260929 16:07:30.641986 992 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:30.642205 992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:30.642594 992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 16:07:30 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:30.662647 992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "6423118d5e1f41929ac07635274b296b"
format_stamp: "Formatted at 2026-09-29 16:07:30 on dist-test-slave-x07d"
server_key: "c0a5cb9f0d894e933d2dd83b36a3ce41"
server_key_iv: "07401143e7dd43a53263c653c2ad0272"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:30.666177 992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "6423118d5e1f41929ac07635274b296b"
format_stamp: "Formatted at 2026-09-29 16:07:30 on dist-test-slave-x07d"
server_key: "c0a5cb9f0d894e933d2dd83b36a3ce41"
server_key_iv: "07401143e7dd43a53263c653c2ad0272"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:30.692222 992 fs_manager.cc:691] Time spent creating directory manager: real 0.025s user 0.009s sys 0.011s
I20260929 16:07:30.717854 1012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:30.723593 992 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.001s
I20260929 16:07:30.723976 992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "6423118d5e1f41929ac07635274b296b"
format_stamp: "Formatted at 2026-09-29 16:07:30 on dist-test-slave-x07d"
server_key: "c0a5cb9f0d894e933d2dd83b36a3ce41"
server_key_iv: "07401143e7dd43a53263c653c2ad0272"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:30.724398 992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:30.772330 992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:30.780805 992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:30.781348 992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:30.783706 992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:30.785601 992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:30.785713 992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:30.785842 992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:30.785905 992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:07:30 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:30.898208 992 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:45795
I20260929 16:07:30.898226 1124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:45795 every 8 connection(s)
I20260929 16:07:30.899760 992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:07:30.909425 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 992
I20260929 16:07:30.909636 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:07:30.910442 20966 external_mini_cluster.cc:1428] Setting key ea8fe1b527a364b91707f2111c89e46b
Sep 29 16:07:30 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:30.941694 1128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:30.905252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:34947 (local address 127.20.121.190:37739)
0929 16:07:30.918493 (+ 13241us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:30.918502 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:30.918529 (+ 27us) server_negotiation.cc:407] Connection header received
0929 16:07:30.920405 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:30.920415 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:30.920535 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:30.920624 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:30.923047 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:30.924211 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:30.926754 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:30.927411 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:30.927525 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:30.933486 (+ 5961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:30.933494 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:30.933499 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:30.933558 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:30.937441 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:30.938596 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:30.938603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:30.938607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:30.938777 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:30.939511 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:30.939516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:30.939520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:30.939888 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:30.940026 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:30.941041 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0929 16:07:30.941293 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13074,"thread_start_us":12928,"threads_started":1}
I20260929 16:07:30.944455 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:30.903481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37739 (local address 127.20.121.129:34947)
0929 16:07:30.914794 (+ 11313us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:30.914863 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:30.917889 (+ 3026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:30.921199 (+ 3310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:30.921213 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:30.921911 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:30.922828 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:30.922855 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:30.924439 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:30.924445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:30.926567 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:30.926596 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:30.927650 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:30.927658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:30.927977 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:30.930249 (+ 2272us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:30.930282 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:30.933140 (+ 2858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:30.937693 (+ 4553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:30.937703 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:30.937726 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:30.938437 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:30.938929 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:30.938934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:30.938937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:30.939352 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:30.940125 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:30.940134 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:30.940595 (+ 461us) client_negotiation.cc:769] Sending connection context
0929 16:07:30.942126 (+ 1531us) client_negotiation.cc:240] Negotiation successful
0929 16:07:30.942799 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10885,"thread_start_us":14941,"threads_started":1}
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:07:30.946625 1125 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37739
I20260929 16:07:30.946954 1125 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:30.947836 1125 heartbeater.cc:511] Master 127.20.121.190:37739 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:07:30.950613 912 ts_manager.cc:194] Registered new tserver with Master: 6423118d5e1f41929ac07635274b296b (127.20.121.129:45795)
I20260929 16:07:30.952700 912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:34947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:07:30.987013 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:47425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37739
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:07:31 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698050, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
W20260929 16:07:31.449335 1131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:31.449620 1131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:31.449785 1131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:31.457563 1131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:31.457773 1131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:31.458057 1131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:07:31.468396 1131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.121.148:47425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:31.469843 1131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:31.471247 1131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:31.484994 1145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:31.484925 1146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:31.484927 1148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:31.485806 1131 server_base.cc:1034] running on GCE node
I20260929 16:07:31.486481 1131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:31.487479 1131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:31.488706 1131 hybrid_clock.cc:648] HybridClock initialized: now 1790698051488643 us; error 58 us; skew 500 ppm
I20260929 16:07:31.489135 1131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:31 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698051, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:31.492453 1131 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:07:31.493901 1131 webserver.cc:460] Webserver started at http://127.20.121.130:44949/ using document root <none> and password file <none>
I20260929 16:07:31.494632 1131 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:31.494733 1131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:31.494995 1131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 16:07:31 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698051, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:31.510392 1131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "c03a5eb6619a4c9bbffc9f90b50caca9"
format_stamp: "Formatted at 2026-09-29 16:07:31 on dist-test-slave-x07d"
server_key: "434b0918460f5d8789aaffcbc087177f"
server_key_iv: "0ec2d5d5bfd66eb301f3787ca4d2d502"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:31.511317 1131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "c03a5eb6619a4c9bbffc9f90b50caca9"
format_stamp: "Formatted at 2026-09-29 16:07:31 on dist-test-slave-x07d"
server_key: "434b0918460f5d8789aaffcbc087177f"
server_key_iv: "0ec2d5d5bfd66eb301f3787ca4d2d502"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:31.517230 1131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 16:07:31.535110 1155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:31.536715 1131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:07:31.536914 1131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "c03a5eb6619a4c9bbffc9f90b50caca9"
format_stamp: "Formatted at 2026-09-29 16:07:31 on dist-test-slave-x07d"
server_key: "434b0918460f5d8789aaffcbc087177f"
server_key_iv: "0ec2d5d5bfd66eb301f3787ca4d2d502"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:31.537153 1131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:31.550937 1131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:31.555006 1131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:31.555323 1131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:31.556414 1131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:31.557857 1131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:31.557964 1131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:31.558132 1131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:31.558203 1131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:31.600816 1131 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:45555
I20260929 16:07:31.600841 1267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:45555 every 8 connection(s)
I20260929 16:07:31.602154 1131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:07:31.610920 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 1131
I20260929 16:07:31.611111 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:07:31.611680 20966 external_mini_cluster.cc:1428] Setting key 696123326c2577ada380d5e1eaad3d55
Sep 29 16:07:31 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698051, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:31.626789 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:31.604969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:57209 (local address 127.20.121.190:37739)
0929 16:07:31.605455 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:31.605462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:31.606651 (+ 1189us) server_negotiation.cc:407] Connection header received
0929 16:07:31.608518 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:31.608523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:31.608617 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:31.608734 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:31.610364 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:31.611349 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:31.613368 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:31.613969 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:31.614127 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:31.619204 (+ 5077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:31.619210 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:31.619213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:31.619259 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:31.622699 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:31.623693 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:31.623703 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:31.623709 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:31.623956 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:31.624467 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:31.624475 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:31.624481 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:31.624934 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:31.625106 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:31.625882 (+ 776us) server_negotiation.cc:299] Negotiation successful
0929 16:07:31.626222 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":221,"threads_started":1}
I20260929 16:07:31.627362 1271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:31.605637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37739 (local address 127.20.121.130:57209)
0929 16:07:31.606478 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:31.606523 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:31.608218 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:31.608921 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:31.608929 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:31.609493 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:31.610172 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:31.610190 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:31.611522 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:31.611528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:31.613227 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:31.613239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:31.614257 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:31.614267 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:31.614617 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:31.615532 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:31.615571 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:31.618868 (+ 3297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:31.622907 (+ 4039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:31.622922 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:31.622947 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:31.623503 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:31.624094 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:31.624099 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:31.624101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:31.624332 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:31.625091 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:31.625099 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:31.625438 (+ 339us) client_negotiation.cc:769] Sending connection context
0929 16:07:31.625680 (+ 242us) client_negotiation.cc:240] Negotiation successful
0929 16:07:31.626163 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":352,"threads_started":1}
I20260929 16:07:31.628930 1268 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37739
I20260929 16:07:31.629295 1268 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:31.630278 1268 heartbeater.cc:511] Master 127.20.121.190:37739 requested a full tablet report, sending...
I20260929 16:07:31.632028 912 ts_manager.cc:194] Registered new tserver with Master: c03a5eb6619a4c9bbffc9f90b50caca9 (127.20.121.130:45555)
I20260929 16:07:31.632838 912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:57209
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:07:31.671492 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:47425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37739
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:07:31.956542 1125 heartbeater.cc:503] Master 127.20.121.190:37739 was elected leader, sending a full tablet report...
W20260929 16:07:32.012408 1274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:32.012769 1274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:32.012931 1274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:32.020601 1274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:32.020778 1274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:32.020954 1274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:07:32.031520 1274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.121.148:47425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:32.033015 1274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:32.034613 1274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:32.048738 1279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:32.048981 1282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:32.049216 1280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:32.050694 1274 server_base.cc:1034] running on GCE node
I20260929 16:07:32.051395 1274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:32.052395 1274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:32.053656 1274 hybrid_clock.cc:648] HybridClock initialized: now 1790698052053599 us; error 49 us; skew 500 ppm
I20260929 16:07:32.054185 1274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:32 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698052, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:32.057657 1274 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:07:32.059572 1274 webserver.cc:460] Webserver started at http://127.20.121.131:43689/ using document root <none> and password file <none>
I20260929 16:07:32.060565 1274 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:32.060732 1274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:32.061069 1274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 16:07:32 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698052, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:32.078814 1274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "0e5990b667514e7a83d825d5371c83f4"
format_stamp: "Formatted at 2026-09-29 16:07:32 on dist-test-slave-x07d"
server_key: "e3dc0a6b017c1731a2624befaa46da15"
server_key_iv: "2cbb23a3394f59c43a4f4455681e949f"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:32.079918 1274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "0e5990b667514e7a83d825d5371c83f4"
format_stamp: "Formatted at 2026-09-29 16:07:32 on dist-test-slave-x07d"
server_key: "e3dc0a6b017c1731a2624befaa46da15"
server_key_iv: "2cbb23a3394f59c43a4f4455681e949f"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:32.086730 1274 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260929 16:07:32.101949 1289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:32.103528 1274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:07:32.103694 1274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "0e5990b667514e7a83d825d5371c83f4"
format_stamp: "Formatted at 2026-09-29 16:07:32 on dist-test-slave-x07d"
server_key: "e3dc0a6b017c1731a2624befaa46da15"
server_key_iv: "2cbb23a3394f59c43a4f4455681e949f"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:32.103948 1274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:32.133935 1274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:32.139025 1274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:32.139386 1274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:32.140453 1274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:32.142212 1274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:32.142324 1274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:32.142485 1274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:32.142560 1274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:32.182379 1274 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:46709
I20260929 16:07:32.182469 1401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:46709 every 8 connection(s)
I20260929 16:07:32.183708 1274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:07:32.187199 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 1274
I20260929 16:07:32.187446 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:07:32.188127 20966 external_mini_cluster.cc:1428] Setting key c9f620412b563d1b884861c5806cf03f
Sep 29 16:07:32 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698052, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:32.215963 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:32.186399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:44799 (local address 127.20.121.190:37739)
0929 16:07:32.186990 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:32.186997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:32.188136 (+ 1139us) server_negotiation.cc:407] Connection header received
0929 16:07:32.190807 (+ 2671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:32.190848 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:32.190976 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:32.191137 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:32.192845 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:32.193604 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:32.195338 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:32.195794 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:32.195898 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:32.209164 (+ 13266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:32.209170 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:32.209174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:32.209220 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:32.212236 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:32.213230 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:32.213235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:32.213237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:32.213412 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:32.213999 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:32.214003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:32.214005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:32.214394 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:32.214539 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:32.215315 (+ 776us) server_negotiation.cc:299] Negotiation successful
0929 16:07:32.215600 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":248,"threads_started":1}
I20260929 16:07:32.217180 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:32.186992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37739 (local address 127.20.121.131:44799)
0929 16:07:32.187936 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:32.187990 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:32.190452 (+ 2462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:32.191430 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:32.191439 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:32.192003 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:32.192682 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:32.192698 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:32.193761 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:32.193765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:32.195184 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:32.195204 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:32.196013 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:32.196018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:32.196233 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:32.206103 (+ 9870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:32.206163 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:32.208856 (+ 2693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:32.212417 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:32.212427 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:32.212444 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:32.213072 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:32.213532 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:32.213536 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:32.213538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:32.213754 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:32.214522 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:32.214528 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:32.214893 (+ 365us) client_negotiation.cc:769] Sending connection context
0929 16:07:32.215189 (+ 296us) client_negotiation.cc:240] Negotiation successful
0929 16:07:32.215710 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":346,"threads_started":1}
I20260929 16:07:32.218868 1402 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37739
I20260929 16:07:32.219237 1402 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:32.220176 1402 heartbeater.cc:511] Master 127.20.121.190:37739 requested a full tablet report, sending...
I20260929 16:07:32.221796 912 ts_manager.cc:194] Registered new tserver with Master: 0e5990b667514e7a83d825d5371c83f4 (127.20.121.131:46709)
I20260929 16:07:32.222728 912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:44799
I20260929 16:07:32.230062 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:07:32.231889 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 992
Sep 29 16:07:32 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:07:32.259377 1407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:32.232889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.20.121.129:45795)
0929 16:07:32.233562 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:32.233574 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:32.233965 (+ 391us) server_negotiation.cc:407] Connection header received
0929 16:07:32.234190 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:32.234198 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:32.234482 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:32.234671 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:32.240218 (+ 5547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:32.242402 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:32.244692 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:32.245763 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:32.245942 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:32.250941 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:32.250961 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:32.250984 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:32.251042 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:32.255528 (+ 4486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:32.256454 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:32.256467 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:32.256483 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:32.256689 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:32.257205 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:32.257212 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:32.257217 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:32.257734 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:32.258138 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:32.258485 (+ 347us) server_negotiation.cc:299] Negotiation successful
0929 16:07:32.258821 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":273,"threads_started":1}
I20260929 16:07:32.261022 1120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:32.348076 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 992
I20260929 16:07:32.365036 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 1131
Sep 29 16:07:32 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:07:32.382424 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:32.366043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.20.121.130:45555)
0929 16:07:32.366630 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:32.366637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:32.366662 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:07:32.366721 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:32.366728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:32.366934 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:32.367138 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:32.368361 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:32.369341 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:32.370730 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:32.371328 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:32.371483 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:32.376329 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:32.376343 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:32.376360 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:32.376408 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:32.379497 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:32.380238 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:32.380246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:32.380248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:32.380386 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:32.380777 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:32.380782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:32.380784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:32.381125 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:32.381419 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:32.381755 (+ 336us) server_negotiation.cc:299] Negotiation successful
0929 16:07:32.382047 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":324,"threads_started":1}
I20260929 16:07:32.383708 1263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:32.468441 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 1131
I20260929 16:07:32.488922 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 1274
Sep 29 16:07:32 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:07:32.506606 1411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:32.489999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51030 (local address 127.20.121.131:46709)
0929 16:07:32.490718 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:32.490725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:32.490755 (+ 30us) server_negotiation.cc:407] Connection header received
0929 16:07:32.490850 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:32.490856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:32.491080 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:32.491260 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:32.492694 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:32.493767 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:32.495173 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:32.495760 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:32.495906 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:32.500278 (+ 4372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:32.500291 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:32.500307 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:32.500350 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:32.503597 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:32.504323 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:32.504331 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:32.504333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:32.504462 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:32.504862 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:32.504866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:32.504868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:32.505255 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:32.505591 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:32.505901 (+ 310us) server_negotiation.cc:299] Negotiation successful
0929 16:07:32.506243 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":313,"threads_started":1}
I20260929 16:07:32.508057 1397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:32.594111 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 1274
I20260929 16:07:32.611349 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 880
I20260929 16:07:32.612730 942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:32.720459 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 880
I20260929 16:07:32.741891 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37739
--webserver_interface=127.20.121.190
--webserver_port=33155
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:37739
--ranger_config_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:47425/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.121.148:47425/kms with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:07:33.082139 1414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:33.082554 1414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:33.082708 1414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:33.090768 1414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:07:33.090956 1414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:33.091058 1414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:33.091161 1414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:07:33.091251 1414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:07:33.102345 1414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.121.148:47425/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:37739
--ranger_config_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=33155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:33.103811 1414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:33.105273 1414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:33.118510 1422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:33.118557 1420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:33.118490 1419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:33.119112 1414 server_base.cc:1034] running on GCE node
I20260929 16:07:33.120088 1414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:33.121698 1414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:33.122995 1414 hybrid_clock.cc:648] HybridClock initialized: now 1790698053122901 us; error 83 us; skew 500 ppm
I20260929 16:07:33.123468 1414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:33 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698053, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:33.127131 1414 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:07:33.129016 1414 webserver.cc:460] Webserver started at http://127.20.121.190:33155/ using document root <none> and password file <none>
I20260929 16:07:33.129822 1414 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:33.130003 1414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 16:07:33 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698053, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:33.187470 1414 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 16:07:33.190935 1430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:33.192531 1414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:07:33.192747 1414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "b2b71ab6b54748df996139a2f634f4b0"
format_stamp: "Formatted at 2026-09-29 16:07:28 on dist-test-slave-x07d"
server_key: "dd22dabf285e882c254125567b9bd6fc"
server_key_iv: "619adcce1d724af1339a0f7bf183f9bf"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:33.193760 1414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:33.215547 1414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:33.220474 1414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:33.220937 1414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:33.239948 1414 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:37739
I20260929 16:07:33.239965 1481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:37739 every 8 connection(s)
I20260929 16:07:33.241539 1414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:07:33.247190 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 1414
I20260929 16:07:33.248080 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:47425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:45795
--local_ip_for_outbound_sockets=127.20.121.129
--tserver_master_addrs=127.20.121.190:37739
--webserver_port=46745
--webserver_interface=127.20.121.129
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.121.148:47425/kms with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:07:33.248895 1482 sys_catalog.cc:263] Verifying existing consensus state
I20260929 16:07:33.252501 1482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: Bootstrap starting.
I20260929 16:07:33.279721 1482 log.cc:824] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:33.291247 1482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 16:07:33.291908 1482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: Bootstrap complete.
I20260929 16:07:33.305454 1482 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } }
I20260929 16:07:33.306787 1482 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b71ab6b54748df996139a2f634f4b0, State: Initialized, Role: FOLLOWER
I20260929 16:07:33.307574 1482 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } }
I20260929 16:07:33.307843 1482 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:07:33.307969 1482 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:07:33.308157 1482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 1 FOLLOWER]: Advancing to term 2
I20260929 16:07:33.312265 1482 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } }
I20260929 16:07:33.312813 1482 leader_election.cc:302] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2b71ab6b54748df996139a2f634f4b0; no voters:
I20260929 16:07:33.313467 1482 leader_election.cc:288] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 16:07:33.313696 1486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 16:07:33.314682 1486 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [term 2 LEADER]: Becoming Leader. State: Replica: b2b71ab6b54748df996139a2f634f4b0, State: Running, Role: LEADER
I20260929 16:07:33.315452 1486 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } }
I20260929 16:07:33.316457 1482 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:07:33.318190 1487 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b2b71ab6b54748df996139a2f634f4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } } }
I20260929 16:07:33.318562 1487 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:33.322394 1488 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2b71ab6b54748df996139a2f634f4b0. Latest consensus state: current_term: 2 leader_uuid: "b2b71ab6b54748df996139a2f634f4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b71ab6b54748df996139a2f634f4b0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37739 } } }
I20260929 16:07:33.323227 1488 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:33.328063 1491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:07:33.333942 1491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:07:33.335485 1491 catalog_manager.cc:1259] Loaded cluster ID: b546e91dcdfe4018945c750a0465f188
I20260929 16:07:33.335732 1491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:07:33.339829 1491 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:07:33.341506 1491 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: Loaded TSK: 0
I20260929 16:07:33.343312 1491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:07:33.345743 1482 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260929 16:07:33.742283 1484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:33.742806 1484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:33.743027 1484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:33.755398 1484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:33.755645 1484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:33.755878 1484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:07:33.772670 1484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.121.148:47425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:45795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=46745
--tserver_master_addrs=127.20.121.190:37739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:33.774900 1484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:33.777200 1484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:33.806003 1519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:33.806056 1520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:33.818619 1484 server_base.cc:1034] running on GCE node
W20260929 16:07:33.818861 1522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:33.819847 1484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:33.824880 1484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:33.826223 1484 hybrid_clock.cc:648] HybridClock initialized: now 1790698053826157 us; error 63 us; skew 500 ppm
I20260929 16:07:33.827605 1484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:33 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698053, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:33.832547 1484 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:07:33.848348 1484 webserver.cc:460] Webserver started at http://127.20.121.129:46745/ using document root <none> and password file <none>
I20260929 16:07:33.849521 1484 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:33.849787 1484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 16:07:33 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698053, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:33.899926 1484 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.004s sys 0.012s
I20260929 16:07:33.915848 1531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:33.918354 1484 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.009s sys 0.004s
I20260929 16:07:33.918604 1484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "6423118d5e1f41929ac07635274b296b"
format_stamp: "Formatted at 2026-09-29 16:07:30 on dist-test-slave-x07d"
server_key: "c0a5cb9f0d894e933d2dd83b36a3ce41"
server_key_iv: "07401143e7dd43a53263c653c2ad0272"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:33.920250 1484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:33.942782 1484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:33.948668 1484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:33.949026 1484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:33.950577 1484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:33.952615 1484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:33.952728 1484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:33.952847 1484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:33.952916 1484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:34.011060 1484 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:45795
I20260929 16:07:34.011129 1643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:45795 every 8 connection(s)
I20260929 16:07:34.013154 1484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:07:34.016822 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 1484
I20260929 16:07:34.017654 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:47425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:45555
--local_ip_for_outbound_sockets=127.20.121.130
--tserver_master_addrs=127.20.121.190:37739
--webserver_port=44949
--webserver_interface=127.20.121.130
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.121.148:47425/kms with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:07:34 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698053, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:34.058718 1647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:34.017652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37739 (local address 127.20.121.129:56251)
0929 16:07:34.029730 (+ 12078us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:34.029796 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:34.031907 (+ 2111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:34.034313 (+ 2406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:34.034328 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:34.035098 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:34.036153 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:34.036187 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:34.039036 (+ 2849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:34.039046 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:34.041699 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:34.041724 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:34.043724 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:34.043735 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:34.044131 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:34.045059 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:34.045100 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:34.048677 (+ 3577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:34.053351 (+ 4674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:34.053363 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:34.053381 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:34.054117 (+ 736us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:34.054725 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:34.054730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:34.054734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:34.054968 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:34.055865 (+ 897us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:34.055874 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:34.056322 (+ 448us) client_negotiation.cc:769] Sending connection context
0929 16:07:34.056636 (+ 314us) client_negotiation.cc:240] Negotiation successful
0929 16:07:34.057248 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11712,"thread_start_us":11466,"threads_started":1}
I20260929 16:07:34.058718 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:34.017458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:56251 (local address 127.20.121.190:37739)
0929 16:07:34.032692 (+ 15234us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:34.032709 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:34.032749 (+ 40us) server_negotiation.cc:407] Connection header received
0929 16:07:34.033171 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:34.033193 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:34.033681 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:34.034149 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:34.036444 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:34.038801 (+ 2357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:34.042042 (+ 3241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:34.043393 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:34.044755 (+ 1362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:34.049062 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:34.049070 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:34.049087 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:34.049132 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:34.053096 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:34.054326 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:34.054333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:34.054339 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:34.054561 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:34.055133 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:34.055138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:34.055142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:34.055711 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:34.055913 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:34.056869 (+ 956us) server_negotiation.cc:299] Negotiation successful
0929 16:07:34.057290 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14964,"thread_start_us":15520,"threads_started":1}
I20260929 16:07:34.062028 1644 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37739
I20260929 16:07:34.062366 1644 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:34.063234 1644 heartbeater.cc:511] Master 127.20.121.190:37739 requested a full tablet report, sending...
W20260929 16:07:34.065969 1644 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:37739 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 16:07:34.066219 1644 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:34.066504 1644 heartbeater.cc:511] Master 127.20.121.190:37739 requested a full tablet report, sending...
I20260929 16:07:34.067595 1644 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:34.067888 1644 heartbeater.cc:511] Master 127.20.121.190:37739 requested a full tablet report, sending...
W20260929 16:07:34.069862 1644 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 16:07:34.477866 1648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:34.478363 1648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:34.478541 1648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:34.491922 1648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:34.492101 1648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:34.492347 1648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:07:34.505378 1648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.121.148:47425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:45555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=44949
--tserver_master_addrs=127.20.121.190:37739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:34.507498 1648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:34.509419 1648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:34.557783 1665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:34.561167 1662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:34.571712 1667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:34.572081 1648 server_base.cc:1034] running on GCE node
I20260929 16:07:34.572961 1648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:34.581141 1648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:34.582484 1648 hybrid_clock.cc:648] HybridClock initialized: now 1790698054582404 us; error 98 us; skew 500 ppm
I20260929 16:07:34.582942 1648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:34 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698054, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:34.588020 1648 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:07:34.595865 1648 webserver.cc:460] Webserver started at http://127.20.121.130:44949/ using document root <none> and password file <none>
I20260929 16:07:34.596979 1648 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:34.597160 1648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 16:07:34 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698054, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:34.627239 1648 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.002s sys 0.009s
Sep 29 16:07:34 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790698054, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:34.630563 1675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:34.632193 1648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 16:07:34.632382 1648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "c03a5eb6619a4c9bbffc9f90b50caca9"
format_stamp: "Formatted at 2026-09-29 16:07:31 on dist-test-slave-x07d"
server_key: "434b0918460f5d8789aaffcbc087177f"
server_key_iv: "0ec2d5d5bfd66eb301f3787ca4d2d502"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:34.633440 1648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:34.678663 1648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:34.683002 1648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:34.683293 1648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:34.684479 1648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:34.685984 1648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:34.686154 1648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260929 16:07:34.686285 1648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:34.686389 1648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:34.739148 1648 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:45555
I20260929 16:07:34.739212 1788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:45555 every 8 connection(s)
I20260929 16:07:34.741458 1648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:07:34.743252 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 1648
I20260929 16:07:34.744014 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.121.148:47425/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:46709
--local_ip_for_outbound_sockets=127.20.121.131
--tserver_master_addrs=127.20.121.190:37739
--webserver_port=43689
--webserver_interface=127.20.121.131
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:33713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.121.148:47425/kms with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:07:34 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698054, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:34.813387 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:34.754831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37739 (local address 127.20.121.130:44849)
0929 16:07:34.780794 (+ 25963us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:34.780847 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:34.783297 (+ 2450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:34.784576 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:34.784590 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:34.785314 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:34.786293 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:34.786325 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:34.787754 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:34.787760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:34.789153 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:34.789177 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:34.790474 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:34.790485 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:34.790753 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:34.791522 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:34.791547 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:34.794082 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:34.807885 (+ 13803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:34.807898 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:34.807927 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:34.808479 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:34.808953 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:34.808959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:34.808964 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:34.809214 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:34.810064 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:34.810073 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:34.810626 (+ 553us) client_negotiation.cc:769] Sending connection context
0929 16:07:34.811011 (+ 385us) client_negotiation.cc:240] Negotiation successful
0929 16:07:34.811659 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25639,"thread_start_us":25428,"threads_started":1}
I20260929 16:07:34.815109 1792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:34.754227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:44849 (local address 127.20.121.190:37739)
0929 16:07:34.780212 (+ 25985us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:34.780217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:34.781013 (+ 796us) server_negotiation.cc:407] Connection header received
0929 16:07:34.783834 (+ 2821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:34.783841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:34.783939 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:34.784054 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:34.786539 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:34.787569 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:34.789353 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:34.790200 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:34.790362 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:34.794336 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:34.794341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:34.794345 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:34.794383 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:34.807641 (+ 13258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:34.808662 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:34.808667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:34.808670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:34.808818 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:34.809356 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:34.809362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:34.809370 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:34.809760 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:34.809934 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:34.814366 (+ 4432us) server_negotiation.cc:299] Negotiation successful
0929 16:07:34.814677 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25842,"thread_start_us":25705,"threads_started":1}
I20260929 16:07:34.816622 1789 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37739
I20260929 16:07:34.816991 1789 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:34.817997 1789 heartbeater.cc:511] Master 127.20.121.190:37739 requested a full tablet report, sending...
I20260929 16:07:34.820442 1447 ts_manager.cc:194] Registered new tserver with Master: c03a5eb6619a4c9bbffc9f90b50caca9 (127.20.121.130:45555)
I20260929 16:07:34.823015 1447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:44849
I20260929 16:07:35.070060 1644 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:35.070518 1644 heartbeater.cc:511] Master 127.20.121.190:37739 requested a full tablet report, sending...
I20260929 16:07:35.071818 1447 ts_manager.cc:194] Registered new tserver with Master: 6423118d5e1f41929ac07635274b296b (127.20.121.129:45795)
I20260929 16:07:35.072896 1447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:56251
Sep 29 16:07:35 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698054, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
W20260929 16:07:35.301146 1791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:35.301599 1791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:35.301765 1791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:35.314749 1791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:35.314956 1791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:35.315160 1791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:07:35.332574 1791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:33713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.121.148:47425/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:46709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=43689
--tserver_master_addrs=127.20.121.190:37739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:35.334446 1791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:35.336243 1791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:35.374908 1799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:35.384802 1798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:35.386471 1802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:35.387284 1791 server_base.cc:1034] running on GCE node
I20260929 16:07:35.388417 1791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:35.395013 1791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:35.396502 1791 hybrid_clock.cc:648] HybridClock initialized: now 1790698055396391 us; error 115 us; skew 500 ppm
I20260929 16:07:35.396944 1791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:35 dist-test-slave-x07d krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698055, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:35.401250 1791 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:07:35.422243 1791 webserver.cc:460] Webserver started at http://127.20.121.131:43689/ using document root <none> and password file <none>
I20260929 16:07:35.423519 1791 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:35.423801 1791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 16:07:35 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698055, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:35.462126 1791 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 16:07:35.472573 1809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:35.475008 1791 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.006s sys 0.004s
I20260929 16:07:35.475346 1791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "0e5990b667514e7a83d825d5371c83f4"
format_stamp: "Formatted at 2026-09-29 16:07:32 on dist-test-slave-x07d"
server_key: "e3dc0a6b017c1731a2624befaa46da15"
server_key_iv: "2cbb23a3394f59c43a4f4455681e949f"
server_key_version: "kuduclusterkey@0"
I20260929 16:07:35.477133 1791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:35.519340 1791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:35.525440 1791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:35.525857 1791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:35.529906 1791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:35.558995 1791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:35.559226 1791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260929 16:07:35.559419 1791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:35.559576 1791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 16:07:35 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698054, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for HTTP/127.20.121.148@KRBTEST.COM
I20260929 16:07:35.796808 1791 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:46709
I20260929 16:07:35.796867 1922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:46709 every 8 connection(s)
I20260929 16:07:35.798621 1791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:07:35.803931 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 1791
Sep 29 16:07:35 dist-test-slave-x07d krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698055, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:35.828441 1789 heartbeater.cc:503] Master 127.20.121.190:37739 was elected leader, sending a full tablet report...
I20260929 16:07:35.832054 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:35.802219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37739 (local address 127.20.121.131:59595)
0929 16:07:35.803088 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:35.803139 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:35.805846 (+ 2707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:35.807143 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:35.807158 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:35.807910 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:35.808900 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:35.808932 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.810623 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.810629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:35.812911 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:35.812942 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.814522 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.814532 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:35.814938 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:35.816452 (+ 1514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:35.816497 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:35.820394 (+ 3897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:35.826371 (+ 5977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:35.826384 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:35.826408 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:35.827164 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:35.827672 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:35.827677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:35.827680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:35.828097 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:35.829022 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:35.829031 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:35.829531 (+ 500us) client_negotiation.cc:769] Sending connection context
0929 16:07:35.829845 (+ 314us) client_negotiation.cc:240] Negotiation successful
0929 16:07:35.830505 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":355,"threads_started":1}
I20260929 16:07:35.833374 1925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:35.801530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:59595 (local address 127.20.121.190:37739)
0929 16:07:35.802061 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:35.802068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:35.803291 (+ 1223us) server_negotiation.cc:407] Connection header received
0929 16:07:35.806392 (+ 3101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:35.806399 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:35.806517 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:35.806648 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:35.809164 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.810399 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.813157 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.813952 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.814140 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:35.820747 (+ 6607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:35.820754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:35.820760 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:35.820840 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:35.824914 (+ 4074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:35.827343 (+ 2429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:35.827360 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:35.827365 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:35.827525 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:35.828250 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:35.828256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:35.828259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:35.828671 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:35.828842 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:35.832636 (+ 3794us) server_negotiation.cc:299] Negotiation successful
0929 16:07:35.832903 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":237,"threads_started":1}
I20260929 16:07:35.835712 1923 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37739
I20260929 16:07:35.836088 1923 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:35.837182 1923 heartbeater.cc:511] Master 127.20.121.190:37739 requested a full tablet report, sending...
I20260929 16:07:35.839810 1447 ts_manager.cc:194] Registered new tserver with Master: 0e5990b667514e7a83d825d5371c83f4 (127.20.121.131:46709)
I20260929 16:07:35.841122 1447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:59595
I20260929 16:07:35.843102 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:35.806065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51626 (local address 127.20.121.190:37739)
0929 16:07:35.818797 (+ 12732us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:35.818806 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:35.818834 (+ 28us) server_negotiation.cc:407] Connection header received
0929 16:07:35.818910 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:35.818916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:35.819026 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:35.819173 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:35.823221 (+ 4048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.824642 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.826826 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.827525 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.827651 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:35.833217 (+ 5566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:35.833224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:35.833228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:35.833270 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:35.838439 (+ 5169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:35.839686 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:35.839692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:35.839696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:35.839894 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:35.840400 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:35.840412 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:35.840416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:35.840767 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:35.841073 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:35.842433 (+ 1360us) server_negotiation.cc:299] Negotiation successful
0929 16:07:35.842686 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12507,"thread_start_us":12318,"threads_started":1}
I20260929 16:07:35.846194 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:07:35.867180 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:35.852391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51636 (local address 127.20.121.190:37739)
0929 16:07:35.852626 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:35.852630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:35.853045 (+ 415us) server_negotiation.cc:407] Connection header received
0929 16:07:35.853253 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:35.853268 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:35.853340 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:35.853438 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:35.855071 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.855800 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.857355 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.857793 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.857900 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:35.860953 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:35.860958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:35.860961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:35.861001 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:35.864105 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:35.864991 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:35.864997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:35.864999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:35.865208 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:35.865695 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:35.865700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:35.865702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:35.866117 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:35.866420 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:35.866675 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 16:07:35.866900 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 16:07:35.872954 1447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:07:35.880769 1447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:07:35.908282 1946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:35.896684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45795 (local address 127.0.0.1:35034)
0929 16:07:35.899017 (+ 2333us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:35.899097 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:35.899297 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:35.900635 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:35.900649 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:35.901047 (+ 398us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:35.901790 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:35.901809 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.903369 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.903375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:35.905624 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:35.905643 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.907188 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.907195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:35.907433 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:35.907480 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 16:07:35.907628 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 16:07:35.907824 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1907,"mutex_wait_us":65,"spinlock_wait_cycles":672128,"thread_start_us":2072,"threads_started":1}
I20260929 16:07:35.909394 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:35.896640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.20.121.129:45795)
0929 16:07:35.897995 (+ 1355us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:35.898039 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:35.900018 (+ 1979us) server_negotiation.cc:407] Connection header received
0929 16:07:35.900132 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:35.900139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:35.900380 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:35.900933 (+ 553us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:35.902076 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.903225 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.905802 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.906904 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.907123 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:35.907326 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:35.907682 (+ 356us) server_negotiation.cc:299] Negotiation successful
0929 16:07:35.909159 (+ 1477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1148,"thread_start_us":956,"threads_started":1}
I20260929 16:07:35.910660 1949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:35.897243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:46709 (local address 127.0.0.1:51040)
0929 16:07:35.899155 (+ 1912us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:35.899175 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:35.899317 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:35.900686 (+ 1369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:35.900691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:35.901047 (+ 356us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:35.901809 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:35.901820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.904229 (+ 2409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.904236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:35.907603 (+ 3367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:35.907620 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.909796 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.909802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:35.909964 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:35.909995 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 16:07:35.910111 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 16:07:35.910239 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1776,"mutex_wait_us":200,"spinlock_wait_cycles":705408,"thread_start_us":1604,"threads_started":1}
I20260929 16:07:35.910758 1948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:35.897790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51040 (local address 127.20.121.131:46709)
0929 16:07:35.898714 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:35.898723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:35.899562 (+ 839us) server_negotiation.cc:407] Connection header received
0929 16:07:35.899718 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:35.899731 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:35.900069 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:35.900326 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:35.902148 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.904018 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.907823 (+ 3805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.909516 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.909723 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:35.909913 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:35.910180 (+ 267us) server_negotiation.cc:299] Negotiation successful
0929 16:07:35.910376 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":579,"threads_started":1}
I20260929 16:07:35.911454 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:35.897738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.20.121.130:45555)
0929 16:07:35.898396 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:35.898405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:35.900876 (+ 2471us) server_negotiation.cc:407] Connection header received
0929 16:07:35.901153 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:35.901161 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:35.901389 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:35.901528 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:35.902384 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.904112 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.907832 (+ 3720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.909516 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.909723 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:35.909913 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:35.910821 (+ 908us) server_negotiation.cc:299] Negotiation successful
0929 16:07:35.911120 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":322,"threads_started":1}
I20260929 16:07:35.912011 1950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:35.898528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:45555 (local address 127.0.0.1:44212)
0929 16:07:35.900496 (+ 1968us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:35.900515 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:35.900949 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:35.901712 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:35.901716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:35.901741 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:35.902168 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:35.902181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.904268 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.904275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:35.907603 (+ 3328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:35.907620 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:35.910133 (+ 2513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:35.910141 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:35.910341 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:35.910379 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 16:07:35.910461 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 16:07:35.910629 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1837,"mutex_wait_us":251,"thread_start_us":1374,"threads_started":1}
I20260929 16:07:35.913635 1579 tablet_service.cc:1467] Processing CreateTablet for tablet be6ab079d6de422c823f17924bf470e6 (DEFAULT_TABLE table=test-table [id=5aa1295c7205472aae679d79aa8648c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:07:35.915307 1579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be6ab079d6de422c823f17924bf470e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:35.914961 1858 tablet_service.cc:1467] Processing CreateTablet for tablet be6ab079d6de422c823f17924bf470e6 (DEFAULT_TABLE table=test-table [id=5aa1295c7205472aae679d79aa8648c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:07:35.916196 1858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be6ab079d6de422c823f17924bf470e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:35.915963 1724 tablet_service.cc:1467] Processing CreateTablet for tablet be6ab079d6de422c823f17924bf470e6 (DEFAULT_TABLE table=test-table [id=5aa1295c7205472aae679d79aa8648c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:07:35.917448 1724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be6ab079d6de422c823f17924bf470e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:35.930537 1951 tablet_bootstrap.cc:492] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: Bootstrap starting.
I20260929 16:07:35.933342 1953 tablet_bootstrap.cc:492] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: Bootstrap starting.
I20260929 16:07:35.935001 1952 tablet_bootstrap.cc:492] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: Bootstrap starting.
I20260929 16:07:35.935132 1951 tablet_bootstrap.cc:654] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:35.937103 1951 log.cc:824] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:35.937943 1953 tablet_bootstrap.cc:654] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:35.938855 1952 tablet_bootstrap.cc:654] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:35.939894 1952 log.cc:824] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:35.939857 1953 log.cc:824] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:35.943272 1953 tablet_bootstrap.cc:492] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: No bootstrap required, opened a new log
I20260929 16:07:35.943508 1952 tablet_bootstrap.cc:492] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: No bootstrap required, opened a new log
I20260929 16:07:35.943604 1953 ts_tablet_manager.cc:1397] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260929 16:07:35.943818 1952 ts_tablet_manager.cc:1397] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 16:07:35.944355 1951 tablet_bootstrap.cc:492] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: No bootstrap required, opened a new log
I20260929 16:07:35.944653 1951 ts_tablet_manager.cc:1397] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260929 16:07:35.952266 1953 raft_consensus.cc:348] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } }
I20260929 16:07:35.952695 1953 raft_consensus.cc:374] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:35.952821 1953 raft_consensus.cc:729] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c03a5eb6619a4c9bbffc9f90b50caca9, State: Initialized, Role: FOLLOWER
I20260929 16:07:35.953482 1953 consensus_queue.cc:260] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } }
I20260929 16:07:35.955961 1952 raft_consensus.cc:348] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } }
I20260929 16:07:35.956563 1952 raft_consensus.cc:374] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:35.956709 1952 raft_consensus.cc:729] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6423118d5e1f41929ac07635274b296b, State: Initialized, Role: FOLLOWER
I20260929 16:07:35.957679 1953 ts_tablet_manager.cc:1428] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260929 16:07:35.957477 1952 consensus_queue.cc:260] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } }
I20260929 16:07:35.956530 1951 raft_consensus.cc:348] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } }
I20260929 16:07:35.958115 1951 raft_consensus.cc:374] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:35.958250 1951 raft_consensus.cc:729] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e5990b667514e7a83d825d5371c83f4, State: Initialized, Role: FOLLOWER
I20260929 16:07:35.958837 1951 consensus_queue.cc:260] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } }
I20260929 16:07:35.959239 1644 heartbeater.cc:503] Master 127.20.121.190:37739 was elected leader, sending a full tablet report...
I20260929 16:07:35.959951 1952 ts_tablet_manager.cc:1428] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: Time spent starting tablet: real 0.016s user 0.015s sys 0.002s
I20260929 16:07:35.962246 1923 heartbeater.cc:503] Master 127.20.121.190:37739 was elected leader, sending a full tablet report...
I20260929 16:07:35.962702 1951 ts_tablet_manager.cc:1428] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: Time spent starting tablet: real 0.018s user 0.019s sys 0.000s
W20260929 16:07:36.000276 1790 tablet.cc:2369] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:07:36.017310 1645 tablet.cc:2369] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:07:36.050554 1924 tablet.cc:2369] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:07:36.288537 1957 raft_consensus.cc:482] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:07:36.288883 1957 raft_consensus.cc:504] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } }
I20260929 16:07:36.290846 1957 leader_election.cc:288] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6423118d5e1f41929ac07635274b296b (127.20.121.129:45795), 0e5990b667514e7a83d825d5371c83f4 (127.20.121.131:46709)
I20260929 16:07:36.301780 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.291301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:51629 (local address 127.20.121.129:45795)
0929 16:07:36.291534 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:36.291541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:36.292431 (+ 890us) server_negotiation.cc:407] Connection header received
0929 16:07:36.292629 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:36.292633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:36.292715 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:36.292827 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:36.293980 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.295145 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.299024 (+ 3879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.300376 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.300561 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.300640 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:36.301299 (+ 659us) server_negotiation.cc:299] Negotiation successful
0929 16:07:36.301462 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 16:07:36.301873 1959 raft_consensus.cc:482] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:07:36.301877 1948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.291405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:50533 (local address 127.20.121.131:46709)
0929 16:07:36.291579 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:36.291584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:36.292430 (+ 846us) server_negotiation.cc:407] Connection header received
0929 16:07:36.292613 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:36.292619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:36.292715 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:36.292829 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:36.293980 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.295202 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.299024 (+ 3822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.300377 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.300561 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.300640 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:36.301291 (+ 651us) server_negotiation.cc:299] Negotiation successful
0929 16:07:36.301462 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 16:07:36.302307 1959 raft_consensus.cc:504] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } }
I20260929 16:07:36.302755 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.291290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45795 (local address 127.20.121.130:51629)
0929 16:07:36.292291 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:36.292307 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:36.292450 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:36.292908 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:36.292916 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:36.292965 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:36.293736 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:36.293762 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.295357 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.295372 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:36.298786 (+ 3414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:36.298811 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.300733 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.300741 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:36.301069 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.301093 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 16:07:36.301194 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 16:07:36.301409 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"mutex_wait_us":352,"spinlock_wait_cycles":27264,"thread_start_us":511,"threads_started":1}
I20260929 16:07:36.302775 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.291187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:46709 (local address 127.20.121.130:50533)
0929 16:07:36.292230 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:36.292251 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:36.292450 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:36.292908 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:36.292916 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:36.292961 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:36.293763 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:36.293775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.295356 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.295372 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:36.298786 (+ 3414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:36.298811 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.300744 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.300749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:36.300995 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.301083 (+ 88us) client_negotiation.cc:769] Sending connection context
0929 16:07:36.301194 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 16:07:36.301409 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"mutex_wait_us":409,"spinlock_wait_cycles":27648,"thread_start_us":606,"threads_started":1}
I20260929 16:07:36.303973 1599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6ab079d6de422c823f17924bf470e6" candidate_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6423118d5e1f41929ac07635274b296b" is_pre_election: true
I20260929 16:07:36.303978 1878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6ab079d6de422c823f17924bf470e6" candidate_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5990b667514e7a83d825d5371c83f4" is_pre_election: true
I20260929 16:07:36.304453 1959 leader_election.cc:288] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6423118d5e1f41929ac07635274b296b (127.20.121.129:45795), c03a5eb6619a4c9bbffc9f90b50caca9 (127.20.121.130:45555)
I20260929 16:07:36.304454 1878 raft_consensus.cc:2454] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c03a5eb6619a4c9bbffc9f90b50caca9 in term 0.
I20260929 16:07:36.304569 1599 raft_consensus.cc:2454] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c03a5eb6619a4c9bbffc9f90b50caca9 in term 0.
I20260929 16:07:36.305461 1678 leader_election.cc:302] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6423118d5e1f41929ac07635274b296b, c03a5eb6619a4c9bbffc9f90b50caca9; no voters:
I20260929 16:07:36.305965 1957 raft_consensus.cc:2781] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:07:36.306141 1957 raft_consensus.cc:482] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:07:36.306262 1957 raft_consensus.cc:3037] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:36.308936 1957 raft_consensus.cc:504] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } }
I20260929 16:07:36.309743 1957 leader_election.cc:288] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [CANDIDATE]: Term 1 election: Requested vote from peers 6423118d5e1f41929ac07635274b296b (127.20.121.129:45795), 0e5990b667514e7a83d825d5371c83f4 (127.20.121.131:46709)
I20260929 16:07:36.310437 1878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6ab079d6de422c823f17924bf470e6" candidate_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e5990b667514e7a83d825d5371c83f4"
I20260929 16:07:36.310454 1599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6ab079d6de422c823f17924bf470e6" candidate_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6423118d5e1f41929ac07635274b296b"
I20260929 16:07:36.310662 1878 raft_consensus.cc:3037] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:36.310703 1599 raft_consensus.cc:3037] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:36.314046 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.304854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:45555 (local address 127.20.121.131:58257)
0929 16:07:36.305196 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:36.305227 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:36.305357 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:36.306995 (+ 1638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:36.307000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:36.307029 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:36.307243 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:36.307253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.308900 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.308908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:36.311339 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:36.311355 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.313185 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.313192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:36.313401 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.313435 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 16:07:36.313532 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 16:07:36.313669 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":97}
I20260929 16:07:36.313988 1878 raft_consensus.cc:2454] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c03a5eb6619a4c9bbffc9f90b50caca9 in term 1.
I20260929 16:07:36.314585 1677 leader_election.cc:302] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e5990b667514e7a83d825d5371c83f4, c03a5eb6619a4c9bbffc9f90b50caca9; no voters:
I20260929 16:07:36.314857 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.305908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:58257 (local address 127.20.121.130:45555)
0929 16:07:36.306594 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:36.306604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:36.306627 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:07:36.306723 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:36.306730 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:36.306829 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:36.306958 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:36.307414 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.308724 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.311542 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.312830 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.313006 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.313086 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:36.314313 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0929 16:07:36.314508 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507}
I20260929 16:07:36.314975 1599 raft_consensus.cc:2454] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c03a5eb6619a4c9bbffc9f90b50caca9 in term 1.
I20260929 16:07:36.314976 1957 raft_consensus.cc:2781] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:07:36.315125 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.305527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:34201 (local address 127.20.121.129:45795)
0929 16:07:36.306003 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:36.306030 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:36.306049 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:07:36.306149 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:36.306158 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:36.306249 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:36.306354 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:36.307140 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.308478 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.311774 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.313483 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.313660 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.313761 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:36.314525 (+ 764us) server_negotiation.cc:299] Negotiation successful
0929 16:07:36.314758 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352}
I20260929 16:07:36.316092 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.305105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45795 (local address 127.20.121.131:34201)
0929 16:07:36.305813 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:36.305833 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:36.305953 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:36.306397 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:36.306405 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:36.306441 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:36.306888 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:36.306907 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.308741 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.308748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:36.311510 (+ 2762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:36.311533 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.313999 (+ 2466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.314038 (+ 39us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:36.314277 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.314318 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 16:07:36.315572 (+ 1254us) client_negotiation.cc:240] Negotiation successful
0929 16:07:36.315755 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":72,"thread_start_us":526,"threads_started":1}
I20260929 16:07:36.315953 1744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6ab079d6de422c823f17924bf470e6" candidate_uuid: "0e5990b667514e7a83d825d5371c83f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" is_pre_election: true
I20260929 16:07:36.316890 1599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be6ab079d6de422c823f17924bf470e6" candidate_uuid: "0e5990b667514e7a83d825d5371c83f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6423118d5e1f41929ac07635274b296b" is_pre_election: true
I20260929 16:07:36.317269 1599 raft_consensus.cc:2379] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0e5990b667514e7a83d825d5371c83f4 in current term 1: Already voted for candidate c03a5eb6619a4c9bbffc9f90b50caca9 in this term.
I20260929 16:07:36.321623 1957 raft_consensus.cc:686] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 1 LEADER]: Becoming Leader. State: Replica: c03a5eb6619a4c9bbffc9f90b50caca9, State: Running, Role: LEADER
I20260929 16:07:36.322235 1957 consensus_queue.cc:237] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } }
I20260929 16:07:36.326965 1810 leader_election.cc:302] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e5990b667514e7a83d825d5371c83f4; no voters: 6423118d5e1f41929ac07635274b296b, c03a5eb6619a4c9bbffc9f90b50caca9
I20260929 16:07:36.327417 1959 raft_consensus.cc:2726] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 16:07:36.328372 1447 catalog_manager.cc:5495] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c03a5eb6619a4c9bbffc9f90b50caca9 (127.20.121.130). New cstate: current_term: 1 leader_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c03a5eb6619a4c9bbffc9f90b50caca9" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 45555 } health_report { overall_health: HEALTHY } } }
I20260929 16:07:36.367812 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.362164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.20.121.130:45555)
0929 16:07:36.362379 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:36.362386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:36.362437 (+ 51us) server_negotiation.cc:407] Connection header received
0929 16:07:36.362661 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:36.362665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:36.362754 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:36.362920 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:07:36.363320 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.364097 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.365511 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.365938 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.366124 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.366649 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:07:36.367184 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:07:36.367353 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:36.367432 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 16:07:36.367515 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 16:07:36.376657 1599 raft_consensus.cc:1261] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [term 1 FOLLOWER]: Refusing update from remote peer c03a5eb6619a4c9bbffc9f90b50caca9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:07:36.376708 1878 raft_consensus.cc:1261] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 1 FOLLOWER]: Refusing update from remote peer c03a5eb6619a4c9bbffc9f90b50caca9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:07:36.377831 1963 consensus_queue.cc:1035] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6423118d5e1f41929ac07635274b296b" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:07:36.378360 1957 consensus_queue.cc:1035] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e5990b667514e7a83d825d5371c83f4" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 46709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:07:36.393142 1966 mvcc.cc:205] Tried to move back new op lower bound from 7334699238907019264 to 7334699238699483136. Current Snapshot: MvccSnapshot[applied={T|T < 7334699238907019264}]
I20260929 16:07:36.393947 1967 mvcc.cc:205] Tried to move back new op lower bound from 7334699238907019264 to 7334699238699483136. Current Snapshot: MvccSnapshot[applied={T|T < 7334699238907019264}]
I20260929 16:07:36.394277 1968 mvcc.cc:205] Tried to move back new op lower bound from 7334699238907019264 to 7334699238699483136. Current Snapshot: MvccSnapshot[applied={T|T < 7334699238907019264}]
I20260929 16:07:36.412839 1447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:07:36.413157 1447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:07:36.417073 1447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2b71ab6b54748df996139a2f634f4b0: Sending DeleteTablet for 3 replicas of tablet be6ab079d6de422c823f17924bf470e6
I20260929 16:07:36.418578 1858 tablet_service.cc:1514] Processing DeleteTablet for tablet be6ab079d6de422c823f17924bf470e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:07:36 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:51040
I20260929 16:07:36.418578 1579 tablet_service.cc:1514] Processing DeleteTablet for tablet be6ab079d6de422c823f17924bf470e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:07:36 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:35034
I20260929 16:07:36.418851 1724 tablet_service.cc:1514] Processing DeleteTablet for tablet be6ab079d6de422c823f17924bf470e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:07:36 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:44212
I20260929 16:07:36.423048 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 1484
I20260929 16:07:36.435067 1975 tablet_replica.cc:330] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: stopping tablet replica
I20260929 16:07:36.435950 1977 tablet_replica.cc:330] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: stopping tablet replica
I20260929 16:07:36.436879 1977 raft_consensus.cc:2229] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:07:36.437568 1975 raft_consensus.cc:2229] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:07:36.437776 1976 tablet_replica.cc:330] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: stopping tablet replica
I20260929 16:07:36.438212 1976 raft_consensus.cc:2229] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:07:36.438373 1975 raft_consensus.cc:2258] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:07:36.438810 1976 raft_consensus.cc:2258] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:07:36.439167 1977 raft_consensus.cc:2258] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:07:36.441083 1976 ts_tablet_manager.cc:1905] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:07:36.441092 1975 ts_tablet_manager.cc:1905] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:07:36.441744 1977 ts_tablet_manager.cc:1905] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:07:36.448586 1976 ts_tablet_manager.cc:1918] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:07:36.448642 1975 ts_tablet_manager.cc:1918] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:07:36.448808 1976 log.cc:1196] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/be6ab079d6de422c823f17924bf470e6
I20260929 16:07:36.449040 1975 log.cc:1196] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/be6ab079d6de422c823f17924bf470e6
I20260929 16:07:36.449031 1977 ts_tablet_manager.cc:1918] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:07:36.449213 1977 log.cc:1196] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/be6ab079d6de422c823f17924bf470e6
I20260929 16:07:36.449388 1976 ts_tablet_manager.cc:1939] T be6ab079d6de422c823f17924bf470e6 P 0e5990b667514e7a83d825d5371c83f4: Deleting consensus metadata
I20260929 16:07:36.449856 1975 ts_tablet_manager.cc:1939] T be6ab079d6de422c823f17924bf470e6 P 6423118d5e1f41929ac07635274b296b: Deleting consensus metadata
I20260929 16:07:36.449885 1977 ts_tablet_manager.cc:1939] T be6ab079d6de422c823f17924bf470e6 P c03a5eb6619a4c9bbffc9f90b50caca9: Deleting consensus metadata
I20260929 16:07:36.451630 1432 catalog_manager.cc:4841] TS 0e5990b667514e7a83d825d5371c83f4 (127.20.121.131:46709): tablet be6ab079d6de422c823f17924bf470e6 (table test-table [id=5aa1295c7205472aae679d79aa8648c6]) successfully deleted
I20260929 16:07:36.451673 1433 catalog_manager.cc:4841] TS 6423118d5e1f41929ac07635274b296b (127.20.121.129:45795): tablet be6ab079d6de422c823f17924bf470e6 (table test-table [id=5aa1295c7205472aae679d79aa8648c6]) successfully deleted
I20260929 16:07:36.453904 1431 catalog_manager.cc:4841] TS c03a5eb6619a4c9bbffc9f90b50caca9 (127.20.121.130:45555): tablet be6ab079d6de422c823f17924bf470e6 (table test-table [id=5aa1295c7205472aae679d79aa8648c6]) successfully deleted
I20260929 16:07:36.456514 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.424255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35036 (local address 127.20.121.129:45795)
0929 16:07:36.424610 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:36.424620 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:36.437990 (+ 13370us) server_negotiation.cc:407] Connection header received
0929 16:07:36.438226 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:36.438231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:36.438355 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:36.438462 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:36.440615 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.441693 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.443677 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.444402 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.444503 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.448431 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:36.448450 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:36.448471 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:36.448522 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:36.452796 (+ 4274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:36.453922 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:36.453933 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:36.453937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:36.454148 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:36.454755 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:36.454760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:36.454764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:36.455226 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:36.455568 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:36.455868 (+ 300us) server_negotiation.cc:299] Negotiation successful
0929 16:07:36.456112 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260929 16:07:36.457238 1638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:36.554293 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 1484
I20260929 16:07:36.573254 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 1648
I20260929 16:07:36.590865 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.574145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44238 (local address 127.20.121.130:45555)
0929 16:07:36.574359 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:36.574365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:36.574389 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:07:36.574553 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:36.574559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:36.574642 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:36.574758 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:36.576165 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.577403 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.578875 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.579597 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.579738 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.583874 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:36.583914 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:36.583953 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:36.584040 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:36.587533 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:36.588611 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:36.588620 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:36.588623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:36.588774 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:36.589224 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:36.589228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:36.589230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:36.589669 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:36.589964 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:36.590315 (+ 351us) server_negotiation.cc:299] Negotiation successful
0929 16:07:36.590563 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 16:07:36.591634 1784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:36.710603 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 1648
I20260929 16:07:36.729585 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 1791
I20260929 16:07:36.744558 1948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:36.730489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51054 (local address 127.20.121.131:46709)
0929 16:07:36.730657 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:36.730661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:36.730682 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:07:36.730907 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:36.730912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:36.730982 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:36.731092 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:36.732448 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.733292 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.734860 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:36.735295 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:36.735423 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:36.738639 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:36.738655 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:36.738671 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:36.738707 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:36.741695 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:36.742523 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:36.742532 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:36.742535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:36.742701 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:36.743075 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:36.743079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:36.743081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:36.743500 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:36.743740 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:36.744058 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 16:07:36.744244 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 16:07:36.745280 1918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:36.838058 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 1791
I20260929 16:07:36.855086 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 1414
I20260929 16:07:36.856321 1477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:36.970745 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 1414
I20260929 16:07:36.994789 20966 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 16:07:37.330683 20966 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 16:07:37.330893 20966 mini_ranger.cc:66] Stopping Ranger...
I20260929 16:07:37.677385 20966 mini_ranger.cc:68] Stopped Ranger
2026-09-29 16:07:37.677 UTC [466] LOG: received smart shutdown request
2026-09-29 16:07:37.681 UTC [466] LOG: background worker "logical replication launcher" (PID 475) exited with exit code 1
2026-09-29 16:07:37.682 UTC [470] LOG: shutting down
2026-09-29 16:07:37.915 UTC [466] LOG: database system is shut down
2026-09-29T16:07:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (58668 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:07:38 dist-test-slave-x07d krb5kdc[1991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:07:38 dist-test-slave-x07d krb5kdc[1991](info): set up 2 sockets
Sep 29 16:07:38 dist-test-slave-x07d krb5kdc[1991](info): commencing operation
krb5kdc: starting...
W20260929 16:07:39.210433 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:07:39 dist-test-slave-x07d krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:07:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:07:39.459057 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:44437
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:44437,127.20.121.189:44519,127.20.121.188:33259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:07:39.808951 2004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:39.809331 2004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:39.809672 2004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:39.817574 2004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:07:39.817802 2004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:39.817919 2004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:39.818046 2004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:07:39.818140 2004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:07:39.829073 2004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:44437,127.20.121.189:44519,127.20.121.188:33259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:44437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:39.830622 2004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:39.832056 2004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:39.845163 2012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:39.845255 2010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:39.845188 2009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:39.846683 2004 server_base.cc:1034] running on GCE node
I20260929 16:07:39.847601 2004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:39.849216 2004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:39.850477 2004 hybrid_clock.cc:648] HybridClock initialized: now 1790698059850424 us; error 62 us; skew 500 ppm
I20260929 16:07:39.850853 2004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:39 dist-test-slave-x07d krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698059, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:39.854175 2004 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:07:39.855715 2004 webserver.cc:460] Webserver started at http://127.20.121.190:42319/ using document root <none> and password file <none>
I20260929 16:07:39.856436 2004 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:39.856539 2004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:39.856802 2004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:07:39.860009 2004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "0fa81b08b7344671ae2389df6ffd2331"
format_stamp: "Formatted at 2026-09-29 16:07:39 on dist-test-slave-x07d"
server_key: "f045e5542396bec8e24afbc3e3e240f7"
server_key_iv: "b7bb8e993cfb0686daf0ee8a0d8dc8fe"
server_key_version: "encryptionkey@0"
I20260929 16:07:39.860877 2004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "0fa81b08b7344671ae2389df6ffd2331"
format_stamp: "Formatted at 2026-09-29 16:07:39 on dist-test-slave-x07d"
server_key: "f045e5542396bec8e24afbc3e3e240f7"
server_key_iv: "b7bb8e993cfb0686daf0ee8a0d8dc8fe"
server_key_version: "encryptionkey@0"
I20260929 16:07:39.866688 2004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 16:07:39.870678 2019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:39.872394 2004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:07:39.872633 2004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "0fa81b08b7344671ae2389df6ffd2331"
format_stamp: "Formatted at 2026-09-29 16:07:39 on dist-test-slave-x07d"
server_key: "f045e5542396bec8e24afbc3e3e240f7"
server_key_iv: "b7bb8e993cfb0686daf0ee8a0d8dc8fe"
server_key_version: "encryptionkey@0"
I20260929 16:07:39.872859 2004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:39.891772 2004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:39.896749 2004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:39.897125 2004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:39.918803 2004 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:44437
I20260929 16:07:39.918825 2070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:44437 every 8 connection(s)
I20260929 16:07:39.920420 2004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:07:39.924669 2071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:39.924827 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 2004
I20260929 16:07:39.925078 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:07:39.925959 20966 external_mini_cluster.cc:1428] Setting key da6fcf7e09bc94e2c860d1e9c9c86add
I20260929 16:07:39.935667 2071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.121.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.189@KRBTEST.COM" created.
Sep 29 16:07:39 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698059, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:39.964406 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:39.939505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46126 (local address 127.20.121.190:44437)
0929 16:07:39.943570 (+ 4065us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:39.943576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:39.943609 (+ 33us) server_negotiation.cc:407] Connection header received
0929 16:07:39.943769 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:39.943775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:39.944062 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:39.944298 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:39.945788 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:39.948036 (+ 2248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:39.949721 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:39.950702 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:39.950858 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:39.955156 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:39.955171 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:39.955192 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:39.955244 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:39.959534 (+ 4290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:39.960309 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:39.960321 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:39.960327 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:39.960579 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:39.961090 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:39.961098 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:39.961104 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:39.961799 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:39.961958 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:39.962838 (+ 880us) server_negotiation.cc:299] Negotiation successful
0929 16:07:39.963177 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":3874,"spinlock_wait_cycles":27008,"thread_start_us":3670,"threads_started":1}
I20260929 16:07:39.964433 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:39.939505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:44437 (local address 127.0.0.1:46126)
0929 16:07:39.940478 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:39.940537 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:39.942766 (+ 2229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:39.944324 (+ 1558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:39.944331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:39.945122 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:39.945495 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:39.945515 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:39.948273 (+ 2758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:39.948278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:39.949518 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:39.949536 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:39.950968 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:39.950974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:39.951106 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:39.951774 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:39.951798 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:39.954848 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:39.959698 (+ 4850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:39.959709 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:39.959720 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:39.960164 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:39.960769 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:39.960773 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:39.960776 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:39.960944 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:39.961994 (+ 1050us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:39.962005 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:39.962348 (+ 343us) client_negotiation.cc:769] Sending connection context
0929 16:07:39.962727 (+ 379us) client_negotiation.cc:240] Negotiation successful
0929 16:07:39.963352 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":56,"thread_start_us":436,"threads_started":1}
I20260929 16:07:39.967921 2071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:07:39.969151 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:39.968601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45050)
0929 16:07:39.968818 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:39.968966 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260929 16:07:39.969451 2020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
I20260929 16:07:39.971457 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:39.970841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45058)
0929 16:07:39.971099 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:39.971206 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260929 16:07:39.971778 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:44519: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab.
Entry for principal kudu/127.20.121.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab.
I20260929 16:07:39.990526 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.189:44519
--webserver_interface=127.20.121.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab
--principal=kudu/127.20.121.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:44437,127.20.121.189:44519,127.20.121.188:33259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:07:40.021242 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } attempt: 1
I20260929 16:07:40.022706 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.022108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45074)
0929 16:07:40.022357 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.022531 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 16:07:40.023874 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.023427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45088)
0929 16:07:40.023593 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.023683 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 16:07:40.024104 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:44519: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
I20260929 16:07:40.092396 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } attempt: 2
I20260929 16:07:40.094065 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.093184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45100)
0929 16:07:40.093637 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.093790 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260929 16:07:40.095329 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.094964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45110)
0929 16:07:40.095124 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.095195 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 16:07:40.095647 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:44519: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
I20260929 16:07:40.186940 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } attempt: 3
I20260929 16:07:40.188215 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.187733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45116)
0929 16:07:40.187953 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.188076 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260929 16:07:40.189661 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.189167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45126)
0929 16:07:40.189326 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.189405 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260929 16:07:40.189985 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:44519: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
I20260929 16:07:40.333338 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } attempt: 4
I20260929 16:07:40.334823 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.334232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45138)
0929 16:07:40.334578 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.334685 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260929 16:07:40.335866 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.335496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45154)
0929 16:07:40.335657 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.335745 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260929 16:07:40.336057 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:44519: Network error: Client connection negotiation failed: client connection to 127.20.121.189:44519: connect: Connection refused (error 111)
W20260929 16:07:40.342525 2078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:40.342916 2078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:40.343073 2078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:40.350870 2078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:07:40.351034 2078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:40.351130 2078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:40.351227 2078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:07:40.351300 2078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:07:40.361764 2078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:44437,127.20.121.189:44519,127.20.121.188:33259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab
--principal=kudu/127.20.121.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.189:44519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.121.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:40.363286 2078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:40.364662 2078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:40.377604 2084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:40.377669 2083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:40.377640 2086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:40.378580 2078 server_base.cc:1034] running on GCE node
I20260929 16:07:40.379452 2078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:40.381081 2078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:40.382332 2078 hybrid_clock.cc:648] HybridClock initialized: now 1790698060382293 us; error 61 us; skew 500 ppm
I20260929 16:07:40.382789 2078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:40 dist-test-slave-x07d krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698060, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:40.386444 2078 init.cc:377] Logged in from keytab as kudu/127.20.121.189@KRBTEST.COM (short username kudu)
I20260929 16:07:40.388237 2078 webserver.cc:460] Webserver started at http://127.20.121.189:40431/ using document root <none> and password file <none>
I20260929 16:07:40.389035 2078 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:40.389189 2078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:40.389508 2078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:07:40.392833 2078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/data/instance:
uuid: "42b596e3e47145ffaa77bc4990e30d18"
format_stamp: "Formatted at 2026-09-29 16:07:40 on dist-test-slave-x07d"
server_key: "4064253cd2f971f67417e90c87c903dd"
server_key_iv: "e37f6dbef1fb589e0c9cbef08b5464d9"
server_key_version: "encryptionkey@0"
I20260929 16:07:40.393721 2078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/wal/instance:
uuid: "42b596e3e47145ffaa77bc4990e30d18"
format_stamp: "Formatted at 2026-09-29 16:07:40 on dist-test-slave-x07d"
server_key: "4064253cd2f971f67417e90c87c903dd"
server_key_iv: "e37f6dbef1fb589e0c9cbef08b5464d9"
server_key_version: "encryptionkey@0"
I20260929 16:07:40.399451 2078 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260929 16:07:40.403294 2093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:40.404908 2078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260929 16:07:40.405120 2078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/wal
uuid: "42b596e3e47145ffaa77bc4990e30d18"
format_stamp: "Formatted at 2026-09-29 16:07:40 on dist-test-slave-x07d"
server_key: "4064253cd2f971f67417e90c87c903dd"
server_key_iv: "e37f6dbef1fb589e0c9cbef08b5464d9"
server_key_version: "encryptionkey@0"
I20260929 16:07:40.405346 2078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:40.439556 2078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:40.444483 2078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:40.444744 2078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:40.465744 2078 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.189:44519
I20260929 16:07:40.465766 2144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.189:44519 every 8 connection(s)
I20260929 16:07:40.467428 2078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
I20260929 16:07:40.471936 2145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:40.474601 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 2078
I20260929 16:07:40.474797 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-1/wal/instance
I20260929 16:07:40.475306 20966 external_mini_cluster.cc:1428] Setting key 6a4e0f16f8d35bdc5e3dc326ade329f7
I20260929 16:07:40.479276 2145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.121.188@KRBTEST.COM; defaulting to no policy
Sep 29 16:07:40 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698060, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.188@KRBTEST.COM" created.
I20260929 16:07:40.506516 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.482132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.20.121.190:44437)
0929 16:07:40.482658 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:40.482664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:40.483825 (+ 1161us) server_negotiation.cc:407] Connection header received
0929 16:07:40.487089 (+ 3264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:40.487094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:40.487182 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:40.487291 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:40.489700 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.490598 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.493382 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.493885 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.494093 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:40.498991 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:40.498997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:40.499001 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:40.499055 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:40.502483 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:40.503634 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:40.503643 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:40.503653 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:40.503814 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:40.504367 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:40.504373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:40.504376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:40.504708 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:40.504891 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:40.505737 (+ 846us) server_negotiation.cc:299] Negotiation successful
0929 16:07:40.506080 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":265,"threads_started":1}
I20260929 16:07:40.507586 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.482729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:44437 (local address 127.0.0.1:46130)
0929 16:07:40.483604 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.483659 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:40.486670 (+ 3011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:40.487607 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:40.487623 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:40.488529 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:40.489450 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:40.489482 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.490792 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.490801 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:40.493154 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:40.493178 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.494220 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.494231 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:40.494605 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:40.495532 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:40.495569 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:40.498573 (+ 3004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:40.502726 (+ 4153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:40.502736 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:40.502754 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:40.503455 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:40.503981 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:40.503985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:40.503987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:40.504211 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:40.504869 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:40.504878 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:40.505306 (+ 428us) client_negotiation.cc:769] Sending connection context
0929 16:07:40.505635 (+ 329us) client_negotiation.cc:240] Negotiation successful
0929 16:07:40.506329 (+ 694us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":338,"threads_started":1}
I20260929 16:07:40.509733 2145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:07:40 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698060, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
I20260929 16:07:40.528470 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.510622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45156)
0929 16:07:40.510852 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.510876 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:40.511004 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:40.512049 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:40.512058 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:40.512634 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:40.512888 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:40.512904 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.514524 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.514534 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:40.516260 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:40.516280 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.517288 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.517301 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:40.517572 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:40.518414 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:40.518445 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:40.521749 (+ 3304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:40.525715 (+ 3966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:40.525720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:40.525723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:40.526226 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:40.526656 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:40.526660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:40.526662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:40.526956 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:40.527589 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:40.527593 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:40.527734 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 16:07:40.527862 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 16:07:40.528150 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 16:07:40.528699 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.510923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45156 (local address 127.20.121.189:44519)
0929 16:07:40.511575 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:40.511582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:40.511609 (+ 27us) server_negotiation.cc:407] Connection header received
0929 16:07:40.511660 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:40.511666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:40.511861 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:40.511983 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:40.513079 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.514360 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.516429 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.517111 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.517219 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:40.522100 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:40.522117 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:40.522134 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:40.522169 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:40.525545 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:40.526373 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:40.526381 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:40.526383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:40.526511 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:40.527076 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:40.527080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:40.527082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:40.527449 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:40.527620 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:40.528098 (+ 478us) server_negotiation.cc:299] Negotiation successful
0929 16:07:40.528313 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":355,"threads_started":1}
I20260929 16:07:40.531459 2145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:07:40.532766 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.532197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:34970)
0929 16:07:40.532448 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.532569 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab.
Entry for principal kudu/127.20.121.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab.
W20260929 16:07:40.533037 2094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
I20260929 16:07:40.534775 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.188:33259
--webserver_interface=127.20.121.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab
--principal=kudu/127.20.121.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:44437,127.20.121.189:44519,127.20.121.188:33259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:07:40.535104 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.534554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:34986)
0929 16:07:40.534757 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.534901 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260929 16:07:40.535400 2145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:33259: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
I20260929 16:07:40.584800 2145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } attempt: 1
I20260929 16:07:40.586254 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.585707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35000)
0929 16:07:40.585915 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.586083 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260929 16:07:40.587558 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.587036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35014)
0929 16:07:40.587257 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.587374 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260929 16:07:40.587805 2145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:33259: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
I20260929 16:07:40.635352 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } attempt: 5
Sep 29 16:07:40 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698059, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
I20260929 16:07:40.650868 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.636306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.20.121.189:44519)
0929 16:07:40.636581 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:40.636588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:40.636610 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:07:40.636867 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:40.636872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:40.636965 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:40.637094 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:40.637993 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.638817 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.640361 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.640846 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.640991 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:40.644667 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:40.644672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:40.644676 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:40.644715 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:40.648010 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:40.648764 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:40.648772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:40.648780 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:40.648905 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:40.649317 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:40.649322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:40.649325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:40.649622 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:40.649792 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:40.650224 (+ 432us) server_negotiation.cc:299] Negotiation successful
0929 16:07:40.650498 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260929 16:07:40.650869 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.636188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45170)
0929 16:07:40.636457 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.636482 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:40.636622 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:40.637139 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:40.637145 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:40.637589 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:40.637804 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:40.637814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.638970 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.638978 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:40.640198 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:40.640212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:40.641031 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:40.641038 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:40.641271 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:40.642110 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:40.642132 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:40.644427 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:40.648196 (+ 3769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:40.648201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:40.648206 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:40.648620 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:40.649026 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:40.649031 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:40.649035 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:40.649184 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:40.649758 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:40.649763 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:40.649924 (+ 161us) client_negotiation.cc:769] Sending connection context
0929 16:07:40.650132 (+ 208us) client_negotiation.cc:240] Negotiation successful
0929 16:07:40.650518 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 16:07:40.652181 2071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:07:40.653331 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.652901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35020)
0929 16:07:40.653074 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.653161 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 16:07:40.654361 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.653980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35024)
0929 16:07:40.654179 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.654244 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260929 16:07:40.654582 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:33259: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
I20260929 16:07:40.656121 2145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } attempt: 2
I20260929 16:07:40.657219 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.656720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35030)
0929 16:07:40.656913 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.657028 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 16:07:40.658456 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.657993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35046)
0929 16:07:40.658209 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.658298 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260929 16:07:40.658713 2145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:33259: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
I20260929 16:07:40.705893 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } attempt: 1
I20260929 16:07:40.707479 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.706922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35050)
0929 16:07:40.707169 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.707303 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260929 16:07:40.708714 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.708329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35066)
0929 16:07:40.708486 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.708569 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 16:07:40.708956 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:33259: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
I20260929 16:07:40.749997 2145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } attempt: 3
I20260929 16:07:40.751361 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.750797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35078)
0929 16:07:40.751039 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.751152 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260929 16:07:40.752661 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.752146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35094)
0929 16:07:40.752422 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.752517 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
W20260929 16:07:40.752969 2145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:33259: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
I20260929 16:07:40.777225 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } attempt: 2
I20260929 16:07:40.778453 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.777925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35104)
0929 16:07:40.778185 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.778287 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260929 16:07:40.779461 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.779084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35120)
0929 16:07:40.779230 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.779350 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 16:07:40.779682 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:33259: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
W20260929 16:07:40.884779 2153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:40.885082 2153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:40.885233 2153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 16:07:40.885923 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } attempt: 3
I20260929 16:07:40.887096 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.886563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35132)
0929 16:07:40.886826 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.886915 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 16:07:40.888093 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.887769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35142)
0929 16:07:40.887911 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.887992 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 16:07:40.888350 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:33259: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
W20260929 16:07:40.893174 2153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:07:40.893270 2153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:40.893339 2153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:40.893440 2153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:07:40.893522 2153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:07:40.896226 2145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } attempt: 4
I20260929 16:07:40.897398 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.896831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35158)
0929 16:07:40.897099 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.897219 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260929 16:07:40.898550 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:40.898164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35166)
0929 16:07:40.898316 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:40.898401 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 16:07:40.898913 2145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:33259: Network error: Client connection negotiation failed: client connection to 127.20.121.188:33259: connect: Connection refused (error 111)
I20260929 16:07:40.904835 2153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:44437,127.20.121.189:44519,127.20.121.188:33259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab
--principal=kudu/127.20.121.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.188:33259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.121.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:40.906385 2153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:40.907747 2153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:40.920271 2159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:40.920245 2161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:40.920588 2158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:40.921433 2153 server_base.cc:1034] running on GCE node
I20260929 16:07:40.922290 2153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:40.923712 2153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:40.924949 2153 hybrid_clock.cc:648] HybridClock initialized: now 1790698060924894 us; error 46 us; skew 500 ppm
I20260929 16:07:40.925289 2153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:40 dist-test-slave-x07d krb5kdc[1991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698060, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:40.928565 2153 init.cc:377] Logged in from keytab as kudu/127.20.121.188@KRBTEST.COM (short username kudu)
I20260929 16:07:40.930160 2153 webserver.cc:460] Webserver started at http://127.20.121.188:41297/ using document root <none> and password file <none>
I20260929 16:07:40.930877 2153 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:40.931025 2153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:40.931324 2153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:07:40.934381 2153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/data/instance:
uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3"
format_stamp: "Formatted at 2026-09-29 16:07:40 on dist-test-slave-x07d"
server_key: "d3c4a46d605c0bed340c448bb9fd947a"
server_key_iv: "75580cb97e21ecfc8bfe235ed29738aa"
server_key_version: "encryptionkey@0"
I20260929 16:07:40.935196 2153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/wal/instance:
uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3"
format_stamp: "Formatted at 2026-09-29 16:07:40 on dist-test-slave-x07d"
server_key: "d3c4a46d605c0bed340c448bb9fd947a"
server_key_iv: "75580cb97e21ecfc8bfe235ed29738aa"
server_key_version: "encryptionkey@0"
I20260929 16:07:40.940547 2153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260929 16:07:40.944300 2168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:40.945819 2153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260929 16:07:40.946092 2153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/wal
uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3"
format_stamp: "Formatted at 2026-09-29 16:07:40 on dist-test-slave-x07d"
server_key: "d3c4a46d605c0bed340c448bb9fd947a"
server_key_iv: "75580cb97e21ecfc8bfe235ed29738aa"
server_key_version: "encryptionkey@0"
I20260929 16:07:40.946318 2153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:40.966223 2153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:40.971154 2153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:40.971446 2153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:40.991091 2153 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.188:33259
I20260929 16:07:40.991137 2219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.188:33259 every 8 connection(s)
I20260929 16:07:40.992565 2153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
I20260929 16:07:40.996620 2220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:40.999078 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 2153
I20260929 16:07:40.999286 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790697841464038-20966-0/minicluster-data/master-2/wal/instance
I20260929 16:07:40.999818 20966 external_mini_cluster.cc:1428] Setting key f9ee8e474a7621c71e266ea193d7be50
I20260929 16:07:41.006109 2220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:07:41 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
Sep 29 16:07:41 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698060, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:41.026312 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.009241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46146 (local address 127.20.121.190:44437)
0929 16:07:41.009951 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.009957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.010554 (+ 597us) server_negotiation.cc:407] Connection header received
0929 16:07:41.010769 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.010775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.010869 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.010995 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.012301 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.013287 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.014865 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.015587 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.015747 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.019551 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.019557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.019560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.019626 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.023000 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.023902 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.023910 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.023913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.024094 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.024563 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.024571 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.024574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.024953 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.025101 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.025654 (+ 553us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.025897 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":395,"threads_started":1}
I20260929 16:07:41.026719 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.001983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35178 (local address 127.20.121.188:33259)
0929 16:07:41.002980 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.003006 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.003067 (+ 61us) server_negotiation.cc:407] Connection header received
0929 16:07:41.005481 (+ 2414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.005511 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.006093 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.006584 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.008169 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.010235 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.011702 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.012638 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.012773 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.017868 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.017884 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.017904 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.017954 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.021531 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.022501 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.022512 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.022527 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.022755 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.023236 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.023240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.023242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.023952 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.024276 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.024802 (+ 526us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.025263 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":373,"threads_started":1}
I20260929 16:07:41.026763 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.009229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:44437 (local address 127.0.0.1:46146)
0929 16:07:41.010414 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:41.010463 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:41.010641 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:41.011058 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:41.011067 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:41.011768 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:41.012134 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:41.012145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.013466 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.013471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:41.014695 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:41.014707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.015840 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.015845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:41.015971 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.016673 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:41.016707 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:41.019307 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:41.023148 (+ 3841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:41.023161 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:41.023171 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:41.023677 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:41.024220 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:41.024225 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:41.024229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:41.024409 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:41.025162 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:41.025168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:41.025342 (+ 174us) client_negotiation.cc:769] Sending connection context
0929 16:07:41.025498 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 16:07:41.025724 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"thread_start_us":658,"threads_started":1}
I20260929 16:07:41.028484 2220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:07:41 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698060, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
Sep 29 16:07:41 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:41.043349 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.029134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:44519 (local address 127.0.0.1:45182)
0929 16:07:41.029293 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:41.029315 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:41.029417 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:41.029841 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:41.029847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:41.030371 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:41.030591 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:41.030600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.031639 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.031644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:41.032683 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:41.032696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.033435 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.033441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:41.033663 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.034338 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:41.034365 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:41.036775 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:41.040597 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:41.040603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:41.040607 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:41.041160 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:41.041572 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:41.041577 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:41.041581 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:41.041741 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:41.042331 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:41.042335 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:41.042505 (+ 170us) client_negotiation.cc:769] Sending connection context
0929 16:07:41.042656 (+ 151us) client_negotiation.cc:240] Negotiation successful
0929 16:07:41.042982 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 16:07:41.043393 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.029357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45182 (local address 127.20.121.189:44519)
0929 16:07:41.029568 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.029572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.029587 (+ 15us) server_negotiation.cc:407] Connection header received
0929 16:07:41.029637 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.029641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.029700 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.029780 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.030741 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.031493 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.032868 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.033284 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.033387 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.037025 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.037034 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.037041 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.037092 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.040447 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.041293 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.041299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.041303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.041452 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.041859 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.041864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.041868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.042212 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.042387 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.042767 (+ 380us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.042989 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 16:07:41.044622 2220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:07:41.045320 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.030146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.20.121.190:44437)
0929 16:07:41.030301 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.030305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.030343 (+ 38us) server_negotiation.cc:407] Connection header received
0929 16:07:41.030540 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.030544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.030616 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.030735 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.032015 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.032677 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.034330 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.034779 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.034879 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.039537 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.039543 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.039546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.039582 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.042421 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.043176 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.043181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.043183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.043308 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.043759 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.043766 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.043769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.044112 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.044417 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.044733 (+ 316us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.044980 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Sep 29 16:07:41 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698060, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
Sep 29 16:07:41 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
I20260929 16:07:41.059343 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.045560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35182 (local address 127.20.121.188:33259)
0929 16:07:41.045705 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.045711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.045728 (+ 17us) server_negotiation.cc:407] Connection header received
0929 16:07:41.045897 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.045902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.045966 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.046096 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.046918 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.047707 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.048802 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.049203 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.049332 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.052799 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.052808 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.052816 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.052859 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.056350 (+ 3491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.057188 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.057193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.057198 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.057350 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.057771 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.057776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.057779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.058184 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.058293 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.058729 (+ 436us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.059042 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":51}
I20260929 16:07:41.059737 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.045344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35182)
0929 16:07:41.045650 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:41.045669 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:41.045772 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:41.046118 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:41.046122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:41.046608 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:41.046748 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:41.046756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.047833 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.047841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:41.048665 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:41.048675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.049340 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.049345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:41.049593 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.050251 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:41.050296 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:41.052586 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:41.056521 (+ 3935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:41.056530 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:41.056538 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:41.057061 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:41.057503 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:41.057507 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:41.057509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:41.057651 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:41.058303 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:41.058311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:41.058484 (+ 173us) client_negotiation.cc:769] Sending connection context
0929 16:07:41.058659 (+ 175us) client_negotiation.cc:240] Negotiation successful
0929 16:07:41.059274 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":257}
I20260929 16:07:41.064312 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.047271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45190 (local address 127.20.121.189:44519)
0929 16:07:41.047437 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.047443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.047597 (+ 154us) server_negotiation.cc:407] Connection header received
0929 16:07:41.047775 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.047781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.047855 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.047955 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.049159 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.050193 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.051586 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.052056 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.052186 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.058146 (+ 5960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.058150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.058154 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.058190 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.061437 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.062353 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.062357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.062359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.062546 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.063022 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.063026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.063028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.063287 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.063518 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.063833 (+ 315us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.064018 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 16:07:41.065593 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } attempt: 4
I20260929 16:07:41.065976 2220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3: Bootstrap starting.
I20260929 16:07:41.069675 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:07:41.070792 2220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:41.072206 2220 log.cc:824] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3: Log is configured to *not* fsync() on all Append() calls
Sep 29 16:07:41 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698059, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
I20260929 16:07:41.075538 2220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3: No bootstrap required, opened a new log
I20260929 16:07:41.083912 2074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.066263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35190)
0929 16:07:41.066440 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:41.066460 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:41.066578 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:41.067167 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:41.067172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:41.067644 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:41.067861 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:41.067871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.069289 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.069295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:41.070668 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:41.070689 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.071781 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.071789 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:41.072002 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.072706 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:41.072733 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:41.076172 (+ 3439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:41.080662 (+ 4490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:41.080668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:41.080673 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:41.081250 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:41.081828 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:41.081833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:41.081837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:41.082028 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:41.082863 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:41.082870 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:41.083053 (+ 183us) client_negotiation.cc:769] Sending connection context
0929 16:07:41.083192 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 16:07:41.083491 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 16:07:41.085435 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.066626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35190 (local address 127.20.121.188:33259)
0929 16:07:41.066838 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.066843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.066860 (+ 17us) server_negotiation.cc:407] Connection header received
0929 16:07:41.066920 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.066924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.066994 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.067097 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.068018 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.069140 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.070841 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.071603 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.071735 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.076522 (+ 4787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.076529 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.076534 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.076601 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.080541 (+ 3940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.081508 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.081513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.081517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.081683 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.082171 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.082178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.082184 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.082608 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.082794 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.084630 (+ 1836us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.084879 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 16:07:41.090981 2220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } }
I20260929 16:07:41.091817 2220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:41.092022 2220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3, State: Initialized, Role: FOLLOWER
I20260929 16:07:41.093139 2220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } }
I20260929 16:07:41.097673 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.076925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.20.121.190:44437)
0929 16:07:41.077111 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.077119 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.077396 (+ 277us) server_negotiation.cc:407] Connection header received
0929 16:07:41.077505 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.077511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.077597 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.077735 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.080204 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.082091 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.084060 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.084686 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.084784 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.089691 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.089701 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.089709 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.089778 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.093856 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.094993 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.095002 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.095008 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.095189 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.095776 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.095784 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.095790 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.096245 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.096511 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.097033 (+ 522us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.097247 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 16:07:41.100577 2235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } } }
I20260929 16:07:41.103919 2235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:07:41.103746 2220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:07:41.105580 2235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:07:41.106634 2235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } }
I20260929 16:07:41.106822 2071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331: Bootstrap starting.
I20260929 16:07:41.111176 2235 leader_election.cc:288] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa81b08b7344671ae2389df6ffd2331 (127.20.121.190:44437), 42b596e3e47145ffaa77bc4990e30d18 (127.20.121.189:44519)
I20260929 16:07:41.111711 2071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:41.113157 2071 log.cc:824] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:41.116840 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.081222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35192 (local address 127.20.121.188:33259)
0929 16:07:41.085678 (+ 4456us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.085684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.098948 (+ 13264us) server_negotiation.cc:407] Connection header received
0929 16:07:41.099213 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.099219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.099310 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.099412 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.101504 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.102644 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.104595 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.105298 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.105411 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.109322 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.109330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.109337 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.109443 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.113190 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.114218 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.114224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.114227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.114434 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.114948 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.114953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.114956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.115323 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.115573 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.116138 (+ 565us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.116435 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":4347,"thread_start_us":19417,"threads_started":1}
I20260929 16:07:41.124936 2071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331: No bootstrap required, opened a new log
I20260929 16:07:41.127628 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.077772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45194 (local address 127.20.121.189:44519)
0929 16:07:41.078793 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.078800 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.081426 (+ 2626us) server_negotiation.cc:407] Connection header received
0929 16:07:41.092186 (+ 10760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.092195 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.092299 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.092376 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.096228 (+ 3852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.097313 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.113929 (+ 16616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.114702 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.114822 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.120019 (+ 5197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.120028 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.120033 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.120080 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.123839 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.124947 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.124954 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.124960 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.125138 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.125642 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.125647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.125650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.126089 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.126363 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.127003 (+ 640us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.127265 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":851}
I20260929 16:07:41.138870 2071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } }
I20260929 16:07:41.139405 2071 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:41.139566 2071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa81b08b7344671ae2389df6ffd2331, State: Initialized, Role: FOLLOWER
I20260929 16:07:41.140290 2071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } }
I20260929 16:07:41.142128 2245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } } }
I20260929 16:07:41.143146 2071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:07:41.143275 2245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:07:41.143180 2046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa81b08b7344671ae2389df6ffd2331" is_pre_election: true
I20260929 16:07:41.143882 2046 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 in term 0.
I20260929 16:07:41.145045 2169 leader_election.cc:302] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa81b08b7344671ae2389df6ffd2331, 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3; no voters:
I20260929 16:07:41.147270 2235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:07:41.147496 2235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:07:41.147595 2235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 0 FOLLOWER]: Advancing to term 1
W20260929 16:07:41.150451 2025 tablet.cc:2369] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:07:41.152490 2235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } }
I20260929 16:07:41.154942 2046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa81b08b7344671ae2389df6ffd2331"
I20260929 16:07:41.155334 2046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:41.158331 2235 leader_election.cc:288] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa81b08b7344671ae2389df6ffd2331 (127.20.121.190:44437), 42b596e3e47145ffaa77bc4990e30d18 (127.20.121.189:44519)
I20260929 16:07:41.160578 2046 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 in term 1.
I20260929 16:07:41.161196 2169 leader_election.cc:302] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa81b08b7344671ae2389df6ffd2331, 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3; no voters:
I20260929 16:07:41.161574 2235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:07:41.173655 2235 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [term 1 LEADER]: Becoming Leader. State: Replica: 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3, State: Running, Role: LEADER
I20260929 16:07:41.174546 2235 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } }
W20260929 16:07:41.193032 2261 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 16:07:41.193181 2261 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 16:07:41.183646 2153 master_runner.cc:186] Error getting master registration for 127.20.121.190:44437: OK, instance_id { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" instance_seqno: 1790698059912729 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 16:07:41.194862 2153 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 16:07:41.195286 2252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3. Latest consensus state: current_term: 1 leader_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } } }
I20260929 16:07:41.195971 2252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:41.198129 2145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } attempt: 5
Sep 29 16:07:41 dist-test-slave-x07d krb5kdc[1991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698060, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
I20260929 16:07:41.191850 2004 master_runner.cc:186] Error getting master registration for 127.20.121.189:44519: OK, instance_id { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" instance_seqno: 1790698060459536 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 16:07:41.211082 2004 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 16:07:41.212486 2251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 16:07:41.212667 2251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 16:07:41.214475 2263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:07:41.216785 2263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:07:41.222312 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.199096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35202 (local address 127.20.121.188:33259)
0929 16:07:41.199250 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:41.199255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:41.199277 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:07:41.199349 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:41.199357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:41.199444 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:41.199561 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:41.200797 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.201924 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.203818 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.204496 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.204611 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.210265 (+ 5654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:41.210273 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:41.210278 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:41.210324 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:41.214259 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.215875 (+ 1616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.215915 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.215919 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.216211 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:41.217268 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:41.217274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:41.217279 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:41.218124 (+ 845us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:41.218442 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:41.219598 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0929 16:07:41.220624 (+ 1026us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 16:07:41.223320 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:41.198728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:33259 (local address 127.0.0.1:35202)
0929 16:07:41.199098 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:41.199119 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:41.199240 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:41.199774 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:41.199781 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:41.200315 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:41.200623 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:41.200638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.202119 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.202125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:41.203637 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:41.203654 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:41.204846 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:41.204852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:41.205064 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:41.205865 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:41.205917 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:41.209944 (+ 4027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:41.214472 (+ 4528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:41.214479 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:41.214484 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:41.215584 (+ 1100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:41.216531 (+ 947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:41.216536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:41.216565 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:41.217065 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:41.218633 (+ 1568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:41.218640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:41.218958 (+ 318us) client_negotiation.cc:769] Sending connection context
0929 16:07:41.220170 (+ 1212us) client_negotiation.cc:240] Negotiation successful
0929 16:07:41.221462 (+ 1292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
W20260929 16:07:41.223658 2174 tablet.cc:2369] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:07:41.230100 2145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18: Bootstrap starting.
I20260929 16:07:41.235023 2046 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [term 1 FOLLOWER]: Refusing update from remote peer 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:07:41.236824 2252 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:07:41.237762 2145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:41.240144 2145 log.cc:824] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:41.257073 2145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18: No bootstrap required, opened a new log
I20260929 16:07:41.263530 2245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3. Latest consensus state: current_term: 1 leader_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } } }
I20260929 16:07:41.263860 2245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:07:41.269696 2245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } } }
I20260929 16:07:41.270031 2245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:07:41.278219 2267 mvcc.cc:205] Tried to move back new op lower bound from 7334699258800205824 to 7334699258578513920. Current Snapshot: MvccSnapshot[applied={T|T < 7334699258800205824}]
I20260929 16:07:41.278270 2145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } }
I20260929 16:07:41.278959 2145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:41.279145 2145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42b596e3e47145ffaa77bc4990e30d18, State: Initialized, Role: FOLLOWER
I20260929 16:07:41.280171 2145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } }
I20260929 16:07:41.286619 2252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } } }
I20260929 16:07:41.286949 2252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:41.287827 2263 catalog_manager.cc:1347] Generated new cluster ID: e0bf8114364440b590f322bca2e53500
I20260929 16:07:41.287931 2263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:07:41.290642 2271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } } }
I20260929 16:07:41.291065 2271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:07:41.293195 2120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42b596e3e47145ffaa77bc4990e30d18" is_pre_election: true
I20260929 16:07:41.293591 2118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:41.294477 2119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42b596e3e47145ffaa77bc4990e30d18"
I20260929 16:07:41.298141 2145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:07:41.298616 2118 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [term 1 FOLLOWER]: Refusing update from remote peer 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:07:41.301534 2235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:07:41.318691 2263 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:07:41.323374 2263 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:07:41.342267 2263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3: Generated new TSK 0
I20260929 16:07:41.343734 2263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:07:41.355470 2271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3. Latest consensus state: current_term: 1 leader_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } } }
I20260929 16:07:41.355829 2271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:07:41.355656 2235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } } }
I20260929 16:07:41.356007 2235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b2af5e6ad2f4e4bbfc47db6c8e8d8e3 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:41.356186 2276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fa81b08b7344671ae2389df6ffd2331" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 44437 } } peers { permanent_uuid: "42b596e3e47145ffaa77bc4990e30d18" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 44519 } } peers { permanent_uuid: "8b2af5e6ad2f4e4bbfc47db6c8e8d8e3" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 33259 } } }
I20260929 16:07:41.356527 2276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:07:41.428966 2290 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 16:07:42.195334 2261 catalog_manager.cc:1259] Loaded cluster ID: e0bf8114364440b590f322bca2e53500
I20260929 16:07:42.195446 2261 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331: loading cluster ID for follower catalog manager: success
I20260929 16:07:42.197857 2261 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331: acquiring CA information for follower catalog manager: success
I20260929 16:07:42.199823 2261 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0fa81b08b7344671ae2389df6ffd2331: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 16:07:42.433434 2290 catalog_manager.cc:1259] Loaded cluster ID: e0bf8114364440b590f322bca2e53500
I20260929 16:07:42.433604 2290 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18: loading cluster ID for follower catalog manager: success
I20260929 16:07:42.436465 2290 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18: acquiring CA information for follower catalog manager: success
I20260929 16:07:42.438683 2290 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 42b596e3e47145ffaa77bc4990e30d18: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 16:07:42.475549 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 2004
I20260929 16:07:42.476815 2066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:42.619354 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 2004
I20260929 16:07:42.641067 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 2078
I20260929 16:07:42.642434 2140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:42.750566 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 2078
I20260929 16:07:42.768962 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 2153
I20260929 16:07:42.770200 2215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:42.880329 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 2153
2026-09-29T16:07:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4833 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:07:42 dist-test-slave-x07d krb5kdc[2297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:07:42 dist-test-slave-x07d krb5kdc[2297](info): set up 2 sockets
Sep 29 16:07:42 dist-test-slave-x07d krb5kdc[2297](info): commencing operation
krb5kdc: starting...
W20260929 16:07:44.034711 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:07:44 dist-test-slave-x07d krb5kdc[2297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:07:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:07:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:07:44.294945 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33613
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:46177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:33613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:07:44.639776 2310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:44.640131 2310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:44.640301 2310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:44.648058 2310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:07:44.648196 2310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:44.648245 2310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:44.648346 2310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:07:44.648416 2310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:07:44.658993 2310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:46177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:33613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:44.660477 2310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:44.661970 2310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:44.675534 2318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:44.675571 2316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:44.675746 2315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:44.676735 2310 server_base.cc:1034] running on GCE node
I20260929 16:07:44.677614 2310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:44.679394 2310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:44.680655 2310 hybrid_clock.cc:648] HybridClock initialized: now 1790698064680607 us; error 44 us; skew 500 ppm
I20260929 16:07:44.681068 2310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:44 dist-test-slave-x07d krb5kdc[2297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698064, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:44.684504 2310 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:07:44.686271 2310 webserver.cc:460] Webserver started at http://127.20.121.190:33551/ using document root <none> and password file <none>
I20260929 16:07:44.687019 2310 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:44.687155 2310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:44.687505 2310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:07:44.690860 2310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "24d861c8ebea450d81938398fff58cf3"
format_stamp: "Formatted at 2026-09-29 16:07:44 on dist-test-slave-x07d"
server_key: "6600331de726d4534e2f13b8e1a3a920"
server_key_iv: "d7b41b68f6522817b365f151b1d88e5a"
server_key_version: "encryptionkey@0"
I20260929 16:07:44.691726 2310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "24d861c8ebea450d81938398fff58cf3"
format_stamp: "Formatted at 2026-09-29 16:07:44 on dist-test-slave-x07d"
server_key: "6600331de726d4534e2f13b8e1a3a920"
server_key_iv: "d7b41b68f6522817b365f151b1d88e5a"
server_key_version: "encryptionkey@0"
I20260929 16:07:44.697769 2310 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260929 16:07:44.702165 2325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:44.704062 2310 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:07:44.704329 2310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "24d861c8ebea450d81938398fff58cf3"
format_stamp: "Formatted at 2026-09-29 16:07:44 on dist-test-slave-x07d"
server_key: "6600331de726d4534e2f13b8e1a3a920"
server_key_iv: "d7b41b68f6522817b365f151b1d88e5a"
server_key_version: "encryptionkey@0"
I20260929 16:07:44.704600 2310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:44.722664 2310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:44.727578 2310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:44.727998 2310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:44.749413 2310 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:33613
I20260929 16:07:44.749420 2376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:33613 every 8 connection(s)
I20260929 16:07:44.751155 2310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:07:44.755363 2377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:44.761215 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 2310
I20260929 16:07:44.761528 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:07:44.762333 20966 external_mini_cluster.cc:1428] Setting key 4c2a1937cd0cfe7964053992cb89830a
I20260929 16:07:44.768299 2377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3: Bootstrap starting.
I20260929 16:07:44.771631 2377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:44.772682 2377 log.cc:824] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:44.775898 2377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3: No bootstrap required, opened a new log
Sep 29 16:07:44 dist-test-slave-x07d krb5kdc[2297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:44.784463 2377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d861c8ebea450d81938398fff58cf3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33613 } }
I20260929 16:07:44.784904 2377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:44.785040 2377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24d861c8ebea450d81938398fff58cf3, State: Initialized, Role: FOLLOWER
I20260929 16:07:44.785760 2377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d861c8ebea450d81938398fff58cf3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33613 } }
I20260929 16:07:44.786049 2377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:07:44.786219 2377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:07:44.786442 2377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:44.787672 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:44.764284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44086 (local address 127.20.121.190:33613)
0929 16:07:44.765280 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:44.765294 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:44.765371 (+ 77us) server_negotiation.cc:407] Connection header received
0929 16:07:44.767392 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:44.767420 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:44.767852 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:44.768276 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:44.769862 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:44.771242 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:44.772742 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:44.773548 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:44.773780 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:44.778177 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:44.778189 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:44.778208 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:44.778254 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:44.782213 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:44.783193 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:44.783206 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:44.783214 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:44.783430 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:44.783889 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:44.783895 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:44.783898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:44.784776 (+ 878us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:44.785152 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:44.785710 (+ 558us) server_negotiation.cc:299] Negotiation successful
0929 16:07:44.786486 (+ 776us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":447,"threads_started":1}
I20260929 16:07:44.789413 2377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d861c8ebea450d81938398fff58cf3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33613 } }
I20260929 16:07:44.789955 2377 leader_election.cc:302] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24d861c8ebea450d81938398fff58cf3; no voters:
I20260929 16:07:44.790763 2377 leader_election.cc:288] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:07:44.790902 2382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:07:44.792371 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:07:44.792399 2382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [term 1 LEADER]: Becoming Leader. State: Replica: 24d861c8ebea450d81938398fff58cf3, State: Running, Role: LEADER
I20260929 16:07:44.792989 2382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d861c8ebea450d81938398fff58cf3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33613 } }
I20260929 16:07:44.793159 2377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:07:44.800206 2384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24d861c8ebea450d81938398fff58cf3. Latest consensus state: current_term: 1 leader_uuid: "24d861c8ebea450d81938398fff58cf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d861c8ebea450d81938398fff58cf3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33613 } } }
I20260929 16:07:44.800192 2383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24d861c8ebea450d81938398fff58cf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24d861c8ebea450d81938398fff58cf3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33613 } } }
I20260929 16:07:44.800658 2384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:44.800734 2383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:44.802434 2389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:07:44.808187 2389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:07:44.818625 2389 catalog_manager.cc:1347] Generated new cluster ID: 6eb9f5440a6341c5bdb71bb1b91eb32a
I20260929 16:07:44.818769 2389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:07:44.832980 2389 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:07:44.834676 2389 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:07:44.845825 2389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24d861c8ebea450d81938398fff58cf3: Generated new TSK 0
I20260929 16:07:44.847093 2389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:07:44.855786 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 2310
I20260929 16:07:44.857206 2372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:44.951449 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 2310
2026-09-29T16:07:44Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2067 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:07:45 dist-test-slave-x07d krb5kdc[2405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:07:45 dist-test-slave-x07d krb5kdc[2405](info): set up 2 sockets
Sep 29 16:07:45 dist-test-slave-x07d krb5kdc[2405](info): commencing operation
krb5kdc: starting...
W20260929 16:07:46.070487 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:07:46 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:07:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:07:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:07:46.325635 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38247
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:38247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:07:46.665905 2418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:46.666363 2418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:46.666540 2418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:46.674410 2418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:07:46.674540 2418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:46.674631 2418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:46.674711 2418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:07:46.674798 2418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:07:46.685410 2418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:38247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:46.686898 2418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:46.688298 2418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:46.700516 2426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:46.700477 2424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:46.700546 2423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:46.701788 2418 server_base.cc:1034] running on GCE node
I20260929 16:07:46.702689 2418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:46.704264 2418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:46.705498 2418 hybrid_clock.cc:648] HybridClock initialized: now 1790698066705429 us; error 54 us; skew 500 ppm
I20260929 16:07:46.705823 2418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:46 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698066, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:46.709311 2418 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:07:46.710989 2418 webserver.cc:460] Webserver started at http://127.20.121.190:32791/ using document root <none> and password file <none>
I20260929 16:07:46.711731 2418 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:46.711838 2418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:46.712210 2418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:07:46.715619 2418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "a133be94e2924db3b07f69b2c6336df1"
format_stamp: "Formatted at 2026-09-29 16:07:46 on dist-test-slave-x07d"
server_key: "1217aa5dfe0420bc6e97ab0e94e87290"
server_key_iv: "7aac5ec5ecb873c553a12a6f77ea63de"
server_key_version: "encryptionkey@0"
I20260929 16:07:46.716498 2418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "a133be94e2924db3b07f69b2c6336df1"
format_stamp: "Formatted at 2026-09-29 16:07:46 on dist-test-slave-x07d"
server_key: "1217aa5dfe0420bc6e97ab0e94e87290"
server_key_iv: "7aac5ec5ecb873c553a12a6f77ea63de"
server_key_version: "encryptionkey@0"
I20260929 16:07:46.722223 2418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260929 16:07:46.725788 2433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:46.727352 2418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:07:46.727587 2418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "a133be94e2924db3b07f69b2c6336df1"
format_stamp: "Formatted at 2026-09-29 16:07:46 on dist-test-slave-x07d"
server_key: "1217aa5dfe0420bc6e97ab0e94e87290"
server_key_iv: "7aac5ec5ecb873c553a12a6f77ea63de"
server_key_version: "encryptionkey@0"
I20260929 16:07:46.727809 2418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:46.740006 2418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:46.744642 2418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:46.744956 2418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:46.764029 2418 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:38247
I20260929 16:07:46.764057 2484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:38247 every 8 connection(s)
I20260929 16:07:46.765533 2418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:07:46.769249 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 2418
I20260929 16:07:46.769477 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:07:46.769717 2485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:46.770313 20966 external_mini_cluster.cc:1428] Setting key 383d8077d42e0a9644bd8124bec258ba
I20260929 16:07:46.784919 2485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Bootstrap starting.
Sep 29 16:07:46 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:46.788337 2485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:46.789440 2485 log.cc:824] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:46.792384 2485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: No bootstrap required, opened a new log
I20260929 16:07:46.793882 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:46.772270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.20.121.190:38247)
0929 16:07:46.773118 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:46.773132 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:46.773174 (+ 42us) server_negotiation.cc:407] Connection header received
0929 16:07:46.774765 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:46.774788 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:46.775251 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:46.775631 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:46.777482 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:46.778821 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:46.780687 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:46.781403 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:46.781545 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:46.786574 (+ 5029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:46.786582 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:46.786597 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:46.786642 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:46.789893 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:46.790677 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:46.790685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:46.790691 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:46.790879 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:46.791269 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:46.791274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:46.791278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:46.791778 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:46.792103 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:46.792522 (+ 419us) server_negotiation.cc:299] Negotiation successful
0929 16:07:46.792954 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":415,"threads_started":1}
I20260929 16:07:46.800580 2485 raft_consensus.cc:348] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:46.801033 2485 raft_consensus.cc:374] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:46.801149 2485 raft_consensus.cc:729] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a133be94e2924db3b07f69b2c6336df1, State: Initialized, Role: FOLLOWER
I20260929 16:07:46.801658 2485 consensus_queue.cc:260] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:46.801867 2485 raft_consensus.cc:388] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:07:46.801963 2485 raft_consensus.cc:482] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:07:46.802134 2485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:46.804616 2485 raft_consensus.cc:504] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:46.805119 2485 leader_election.cc:302] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a133be94e2924db3b07f69b2c6336df1; no voters:
I20260929 16:07:46.805689 2485 leader_election.cc:288] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:07:46.805821 2490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:07:46.806883 2490 raft_consensus.cc:686] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 1 LEADER]: Becoming Leader. State: Replica: a133be94e2924db3b07f69b2c6336df1, State: Running, Role: LEADER
I20260929 16:07:46.807487 2490 consensus_queue.cc:237] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:46.808140 2485 sys_catalog.cc:564] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:07:46.810534 2491 sys_catalog.cc:455] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a133be94e2924db3b07f69b2c6336df1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } } }
I20260929 16:07:46.810902 2491 sys_catalog.cc:458] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:46.811444 2492 sys_catalog.cc:455] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a133be94e2924db3b07f69b2c6336df1. Latest consensus state: current_term: 1 leader_uuid: "a133be94e2924db3b07f69b2c6336df1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } } }
I20260929 16:07:46.811900 2492 sys_catalog.cc:458] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:46.811951 2497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:07:46.818540 2497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:07:46.828749 2497 catalog_manager.cc:1347] Generated new cluster ID: 25a803d4827c4190a6383d9b85019e59
I20260929 16:07:46.828954 2497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:07:46.838898 2497 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:07:46.840276 2497 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:07:46.847227 2497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Generated new TSK 0
I20260929 16:07:46.848232 2497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:07:46.914191 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38247
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:07:47.261185 2511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:47.261476 2511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:47.261646 2511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:47.269346 2511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:47.269470 2511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:47.269680 2511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:07:47.279966 2511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:47.281394 2511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:47.282843 2511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:47.297811 2516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:47.297973 2517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:47.298408 2519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:47.299960 2511 server_base.cc:1034] running on GCE node
I20260929 16:07:47.300655 2511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:47.301680 2511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:47.302953 2511 hybrid_clock.cc:648] HybridClock initialized: now 1790698067302856 us; error 82 us; skew 500 ppm
I20260929 16:07:47.303349 2511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:47 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698067, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:47.319967 2511 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:07:47.322237 2511 webserver.cc:460] Webserver started at http://127.20.121.129:33953/ using document root <none> and password file <none>
I20260929 16:07:47.323299 2511 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:47.323465 2511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:47.323853 2511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:07:47.328622 2511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "18eb6e2c53b343fe8b3054da835182d0"
format_stamp: "Formatted at 2026-09-29 16:07:47 on dist-test-slave-x07d"
server_key: "204228a45c98fa2fdd2b842602ccdfc1"
server_key_iv: "289663eaf442825eda69925ca48320d3"
server_key_version: "encryptionkey@0"
I20260929 16:07:47.329900 2511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "18eb6e2c53b343fe8b3054da835182d0"
format_stamp: "Formatted at 2026-09-29 16:07:47 on dist-test-slave-x07d"
server_key: "204228a45c98fa2fdd2b842602ccdfc1"
server_key_iv: "289663eaf442825eda69925ca48320d3"
server_key_version: "encryptionkey@0"
I20260929 16:07:47.337553 2511 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260929 16:07:47.342510 2526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:47.344498 2511 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260929 16:07:47.344722 2511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "18eb6e2c53b343fe8b3054da835182d0"
format_stamp: "Formatted at 2026-09-29 16:07:47 on dist-test-slave-x07d"
server_key: "204228a45c98fa2fdd2b842602ccdfc1"
server_key_iv: "289663eaf442825eda69925ca48320d3"
server_key_version: "encryptionkey@0"
I20260929 16:07:47.344985 2511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:47.373909 2511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:47.378443 2511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:47.378769 2511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:47.379869 2511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:47.381389 2511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:47.381466 2511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:47.381593 2511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:47.381664 2511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:47.416656 2511 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:45329
I20260929 16:07:47.416772 2638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:45329 every 8 connection(s)
I20260929 16:07:47.418291 2511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:07:47.418987 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 2511
I20260929 16:07:47.419270 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:07:47.420033 20966 external_mini_cluster.cc:1428] Setting key 0a68028e76b2d005f701ae0c28e6f5eb
Sep 29 16:07:47 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698067, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:47.446666 2641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:47.421455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:52783 (local address 127.20.121.190:38247)
0929 16:07:47.421973 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:47.421979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:47.423290 (+ 1311us) server_negotiation.cc:407] Connection header received
0929 16:07:47.426909 (+ 3619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:47.426914 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:47.427024 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:47.427141 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:47.429730 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:47.430714 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:47.432834 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:47.433705 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:47.433881 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:47.438310 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:47.438321 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:47.438330 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:47.438396 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:47.442639 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:47.443509 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:47.443519 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:47.443525 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:47.443700 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:47.444210 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:47.444220 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:47.444227 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:47.444699 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:47.444886 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:47.445716 (+ 830us) server_negotiation.cc:299] Negotiation successful
0929 16:07:47.446087 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":265,"threads_started":1}
I20260929 16:07:47.447245 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:47.422116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:52783)
0929 16:07:47.423093 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:47.423145 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:47.426461 (+ 3316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:47.427511 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:47.427528 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:47.428502 (+ 974us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:47.429474 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:47.429504 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:47.430943 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:47.430953 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:47.432654 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:47.432668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:47.433989 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:47.433995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:47.434391 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:47.435202 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:47.435227 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:47.437950 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:47.442822 (+ 4872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:47.442831 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:47.442847 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:47.443358 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:47.443823 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:47.443826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:47.443829 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:47.444056 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:47.444851 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:47.444857 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:47.445257 (+ 400us) client_negotiation.cc:769] Sending connection context
0929 16:07:47.445511 (+ 254us) client_negotiation.cc:240] Negotiation successful
0929 16:07:47.445937 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":381,"threads_started":1}
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:07:47.449159 2639 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38247
I20260929 16:07:47.449487 2639 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:47.450413 2639 heartbeater.cc:511] Master 127.20.121.190:38247 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:07:47.453142 2450 ts_manager.cc:194] Registered new tserver with Master: 18eb6e2c53b343fe8b3054da835182d0 (127.20.121.129:45329)
I20260929 16:07:47.455411 2450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:52783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:07:47.481845 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38247
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:07:47.818279 2645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:47.818607 2645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:47.818799 2645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:47.826378 2645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:47.826539 2645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:47.826731 2645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:07:47.837036 2645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:47.838455 2645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:47.839855 2645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:47.853889 2650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:47.853861 2651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:47.853937 2653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:47.859851 2645 server_base.cc:1034] running on GCE node
I20260929 16:07:47.860616 2645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:47.861718 2645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:47.862991 2645 hybrid_clock.cc:648] HybridClock initialized: now 1790698067862904 us; error 84 us; skew 500 ppm
I20260929 16:07:47.863356 2645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:47 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698067, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:47.866668 2645 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:07:47.868270 2645 webserver.cc:460] Webserver started at http://127.20.121.130:38903/ using document root <none> and password file <none>
I20260929 16:07:47.868963 2645 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:47.869061 2645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:47.869309 2645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:07:47.872311 2645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "6eaf9b43fcec4d849eb22385dd22f1a8"
format_stamp: "Formatted at 2026-09-29 16:07:47 on dist-test-slave-x07d"
server_key: "779435239a1a9e3f56ca6c892fd15b69"
server_key_iv: "dc980135bc3a5f29be31df9655eaf1bc"
server_key_version: "encryptionkey@0"
I20260929 16:07:47.873104 2645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "6eaf9b43fcec4d849eb22385dd22f1a8"
format_stamp: "Formatted at 2026-09-29 16:07:47 on dist-test-slave-x07d"
server_key: "779435239a1a9e3f56ca6c892fd15b69"
server_key_iv: "dc980135bc3a5f29be31df9655eaf1bc"
server_key_version: "encryptionkey@0"
I20260929 16:07:47.878545 2645 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 16:07:47.882285 2660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:47.883733 2645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:07:47.883939 2645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "6eaf9b43fcec4d849eb22385dd22f1a8"
format_stamp: "Formatted at 2026-09-29 16:07:47 on dist-test-slave-x07d"
server_key: "779435239a1a9e3f56ca6c892fd15b69"
server_key_iv: "dc980135bc3a5f29be31df9655eaf1bc"
server_key_version: "encryptionkey@0"
I20260929 16:07:47.884161 2645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:47.916328 2645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:47.920907 2645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:47.921202 2645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:47.922297 2645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:47.923638 2645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:47.923712 2645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:47.923870 2645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:47.923939 2645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:47.958571 2645 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:38141
I20260929 16:07:47.958658 2772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:38141 every 8 connection(s)
I20260929 16:07:47.959918 2645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:07:47.966483 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 2645
I20260929 16:07:47.966681 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:07:47.967172 20966 external_mini_cluster.cc:1428] Setting key 5dbe1f09b030b4157ce046a305fb7143
Sep 29 16:07:47 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698067, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:47.984427 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:47.962583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:38657 (local address 127.20.121.190:38247)
0929 16:07:47.963056 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:47.963061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:47.963905 (+ 844us) server_negotiation.cc:407] Connection header received
0929 16:07:47.965718 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:47.965723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:47.965813 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:47.965940 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:47.967562 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:47.968410 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:47.970133 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:47.970636 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:47.970771 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:47.978499 (+ 7728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:47.978505 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:47.978509 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:47.978544 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:47.981133 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:47.981916 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:47.981926 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:47.981929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:47.982115 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:47.982574 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:47.982579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:47.982583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:47.982876 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:47.983036 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:47.983794 (+ 758us) server_negotiation.cc:299] Negotiation successful
0929 16:07:47.984041 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":248,"threads_started":1}
I20260929 16:07:47.985538 2776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:47.963082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:38657)
0929 16:07:47.963747 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:47.963791 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:47.965456 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:47.966199 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:47.966207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:47.966791 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:47.967391 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:47.967409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:47.968564 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:47.968571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:47.969957 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:47.969968 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:47.970857 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:47.970866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:47.971109 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:47.975550 (+ 4441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:47.975587 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:47.978222 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:47.981301 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:47.981311 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:47.981330 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:47.981771 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:47.982230 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:47.982235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:47.982238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:47.982450 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:47.983006 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:47.983015 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:47.983400 (+ 385us) client_negotiation.cc:769] Sending connection context
0929 16:07:47.983708 (+ 308us) client_negotiation.cc:240] Negotiation successful
0929 16:07:47.984260 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":278,"threads_started":1}
I20260929 16:07:47.987118 2773 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38247
I20260929 16:07:47.987440 2773 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:47.988265 2773 heartbeater.cc:511] Master 127.20.121.190:38247 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
I20260929 16:07:47.989918 2450 ts_manager.cc:194] Registered new tserver with Master: 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141)
I20260929 16:07:47.990864 2450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:38657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:07:48.025775 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38247
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:07:48.375748 2779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:48.376114 2779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:48.376286 2779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:48.384073 2779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:48.384263 2779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:48.384465 2779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:07:48.395324 2779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:48.396766 2779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:48.398301 2779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:48.413178 2785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:48.413254 2787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:48.413201 2784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:48.414258 2779 server_base.cc:1034] running on GCE node
I20260929 16:07:48.415052 2779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:48.416402 2779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:48.417713 2779 hybrid_clock.cc:648] HybridClock initialized: now 1790698068417609 us; error 86 us; skew 500 ppm
I20260929 16:07:48.418171 2779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:48 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698068, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:48.422621 2779 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:07:48.424317 2779 webserver.cc:460] Webserver started at http://127.20.121.131:38357/ using document root <none> and password file <none>
I20260929 16:07:48.425084 2779 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:48.425220 2779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:48.425554 2779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:07:48.428712 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "62cbbbd02f9447dca873969bee33ca24"
format_stamp: "Formatted at 2026-09-29 16:07:48 on dist-test-slave-x07d"
server_key: "c6ce0e1167e46f3953e2934939af69d9"
server_key_iv: "7342659e2deb3325ba505eb583101dd4"
server_key_version: "encryptionkey@0"
I20260929 16:07:48.429672 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "62cbbbd02f9447dca873969bee33ca24"
format_stamp: "Formatted at 2026-09-29 16:07:48 on dist-test-slave-x07d"
server_key: "c6ce0e1167e46f3953e2934939af69d9"
server_key_iv: "7342659e2deb3325ba505eb583101dd4"
server_key_version: "encryptionkey@0"
I20260929 16:07:48.435496 2779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 16:07:48.439313 2794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:48.440937 2779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:07:48.441131 2779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "62cbbbd02f9447dca873969bee33ca24"
format_stamp: "Formatted at 2026-09-29 16:07:48 on dist-test-slave-x07d"
server_key: "c6ce0e1167e46f3953e2934939af69d9"
server_key_iv: "7342659e2deb3325ba505eb583101dd4"
server_key_version: "encryptionkey@0"
I20260929 16:07:48.441350 2779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:48.455847 2779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:48.459069 2639 heartbeater.cc:503] Master 127.20.121.190:38247 was elected leader, sending a full tablet report...
I20260929 16:07:48.460619 2779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:48.460942 2779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:48.462167 2779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:48.463604 2779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:48.463711 2779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:48.463876 2779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:48.463955 2779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:48.500550 2779 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:43105
I20260929 16:07:48.500665 2906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:43105 every 8 connection(s)
I20260929 16:07:48.501943 2779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:07:48.510653 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 2779
I20260929 16:07:48.510869 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:07:48.511345 20966 external_mini_cluster.cc:1428] Setting key ece4243b4dce451379c8b963138543f3
Sep 29 16:07:48 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698068, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:48.525363 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.504721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:51701 (local address 127.20.121.190:38247)
0929 16:07:48.505196 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:48.505201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:48.506300 (+ 1099us) server_negotiation.cc:407] Connection header received
0929 16:07:48.508295 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:48.508302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:48.508403 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:48.508528 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:48.510258 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.511164 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.513348 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.513842 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.513956 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.518193 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:48.518199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:48.518212 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:48.518256 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:48.521575 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:48.522676 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:48.522682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:48.522684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:48.522828 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:48.523458 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:48.523463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:48.523465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:48.523787 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:48.523929 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:48.524748 (+ 819us) server_negotiation.cc:299] Negotiation successful
0929 16:07:48.524985 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":231,"threads_started":1}
I20260929 16:07:48.526932 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.505295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:51701)
0929 16:07:48.506125 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:48.506173 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:48.507989 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:48.508785 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:48.508795 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:48.509407 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:48.510077 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.510098 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.511332 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.511336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.513124 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.513142 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.514115 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.514123 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.514514 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.515221 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:48.515245 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:48.517692 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:48.521814 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:48.521824 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:48.521840 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:48.522515 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:48.522960 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:48.522963 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:48.522966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:48.523175 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:48.523936 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:48.523943 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:48.524332 (+ 389us) client_negotiation.cc:769] Sending connection context
0929 16:07:48.524601 (+ 269us) client_negotiation.cc:240] Negotiation successful
0929 16:07:48.525311 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":284,"threads_started":1}
I20260929 16:07:48.528515 2907 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38247
I20260929 16:07:48.528852 2907 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:48.529637 2907 heartbeater.cc:511] Master 127.20.121.190:38247 requested a full tablet report, sending...
I20260929 16:07:48.531276 2450 ts_manager.cc:194] Registered new tserver with Master: 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105)
I20260929 16:07:48.532274 2450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:51701
I20260929 16:07:48.540056 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:07:48.558876 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.544715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41690 (local address 127.20.121.190:38247)
0929 16:07:48.545035 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:48.545041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:48.545284 (+ 243us) server_negotiation.cc:407] Connection header received
0929 16:07:48.545497 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:48.545500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:48.545573 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:48.545659 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:48.547328 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.548091 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.549485 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.549924 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.550139 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.552660 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:48.552665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:48.552669 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:48.552707 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:48.555718 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:48.556580 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:48.556585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:48.556587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:48.556744 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:48.557252 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:48.557256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:48.557258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:48.557613 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:48.557811 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:48.558323 (+ 512us) server_negotiation.cc:299] Negotiation successful
0929 16:07:48.558533 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 16:07:48.563838 2450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:07:48.571141 2450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:07:48.598207 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.585328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:38141 (local address 127.0.0.1:55442)
0929 16:07:48.588399 (+ 3071us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:48.588436 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:48.588755 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:48.589970 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:48.589974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:48.590001 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:48.590603 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.590615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.592385 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.592394 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.595156 (+ 2762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.595175 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.597203 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.597209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.597398 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.597433 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 16:07:48.597500 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 16:07:48.597640 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2076,"mutex_wait_us":39,"thread_start_us":2960,"threads_started":1}
I20260929 16:07:48.597870 2920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.585811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:43105 (local address 127.0.0.1:48696)
0929 16:07:48.588731 (+ 2920us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:48.588749 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:48.588885 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:48.589761 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:48.589772 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:48.589803 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:48.590448 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.590468 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.593086 (+ 2618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.593092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.595348 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.595360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.596740 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.596747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.596959 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.597021 (+ 62us) client_negotiation.cc:769] Sending connection context
0929 16:07:48.597157 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 16:07:48.597388 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2786,"mutex_wait_us":25,"thread_start_us":1461,"threads_started":1}
I20260929 16:07:48.599211 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.586493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48696 (local address 127.20.121.131:43105)
0929 16:07:48.587969 (+ 1476us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:48.587977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:48.589054 (+ 1077us) server_negotiation.cc:407] Connection header received
0929 16:07:48.589167 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:48.589174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:48.589445 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:48.590744 (+ 1299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:48.591270 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.592915 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.595519 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.596507 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.596740 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.596959 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:48.598832 (+ 1873us) server_negotiation.cc:299] Negotiation successful
0929 16:07:48.598983 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1309,"thread_start_us":1145,"threads_started":1}
I20260929 16:07:48.599534 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.585569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55442 (local address 127.20.121.130:38141)
0929 16:07:48.587218 (+ 1649us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:48.587242 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:48.588646 (+ 1404us) server_negotiation.cc:407] Connection header received
0929 16:07:48.589045 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:48.589057 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:48.589430 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:48.589647 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:48.590789 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.592224 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.595380 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.596949 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.597986 (+ 1037us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.598208 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:48.598755 (+ 547us) server_negotiation.cc:299] Negotiation successful
0929 16:07:48.598957 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1457,"thread_start_us":1280,"threads_started":1}
I20260929 16:07:48.599840 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.589949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59814)
0929 16:07:48.591155 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:48.591174 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:48.591321 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:48.592012 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:48.592016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:48.592042 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:48.592478 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.592523 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.594438 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.594447 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.596524 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.596540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.598905 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.598911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.599090 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.599150 (+ 60us) client_negotiation.cc:769] Sending connection context
0929 16:07:48.599251 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 16:07:48.599413 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1093,"mutex_wait_us":1,"thread_start_us":946,"threads_started":1}
I20260929 16:07:48.599880 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.589164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59814 (local address 127.20.121.129:45329)
0929 16:07:48.590525 (+ 1361us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:48.590534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:48.591302 (+ 768us) server_negotiation.cc:407] Connection header received
0929 16:07:48.591495 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:48.591506 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:48.591790 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:48.592005 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:48.592758 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.594282 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.596752 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.598392 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.598583 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.598763 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:48.599342 (+ 579us) server_negotiation.cc:299] Negotiation successful
0929 16:07:48.599554 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1178,"thread_start_us":1015,"threads_started":1}
I20260929 16:07:48.603552 2574 tablet_service.cc:1467] Processing CreateTablet for tablet 01e3e51659d9430881f77746fe654e66 (DEFAULT_TABLE table=test-table [id=a8a9c8b01aba4b848df9cf9f95a06170]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:07:48.603552 2708 tablet_service.cc:1467] Processing CreateTablet for tablet 01e3e51659d9430881f77746fe654e66 (DEFAULT_TABLE table=test-table [id=a8a9c8b01aba4b848df9cf9f95a06170]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:07:48.603552 2842 tablet_service.cc:1467] Processing CreateTablet for tablet 01e3e51659d9430881f77746fe654e66 (DEFAULT_TABLE table=test-table [id=a8a9c8b01aba4b848df9cf9f95a06170]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:07:48.604983 2574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01e3e51659d9430881f77746fe654e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:48.605013 2842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01e3e51659d9430881f77746fe654e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:48.605020 2708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01e3e51659d9430881f77746fe654e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:48.618840 2926 tablet_bootstrap.cc:492] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Bootstrap starting.
I20260929 16:07:48.621091 2927 tablet_bootstrap.cc:492] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: Bootstrap starting.
I20260929 16:07:48.622264 2928 tablet_bootstrap.cc:492] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: Bootstrap starting.
I20260929 16:07:48.623113 2926 tablet_bootstrap.cc:654] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:48.624545 2926 log.cc:824] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:48.625488 2928 tablet_bootstrap.cc:654] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:48.625722 2927 tablet_bootstrap.cc:654] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:48.626937 2928 log.cc:824] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:48.626997 2927 log.cc:824] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:48.628667 2926 tablet_bootstrap.cc:492] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: No bootstrap required, opened a new log
I20260929 16:07:48.629002 2926 ts_tablet_manager.cc:1397] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260929 16:07:48.629914 2927 tablet_bootstrap.cc:492] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: No bootstrap required, opened a new log
I20260929 16:07:48.630201 2928 tablet_bootstrap.cc:492] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: No bootstrap required, opened a new log
I20260929 16:07:48.630224 2927 ts_tablet_manager.cc:1397] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260929 16:07:48.630470 2928 ts_tablet_manager.cc:1397] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 16:07:48.640107 2927 raft_consensus.cc:348] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } }
I20260929 16:07:48.640599 2927 raft_consensus.cc:374] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:48.640296 2926 raft_consensus.cc:348] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } }
I20260929 16:07:48.640722 2927 raft_consensus.cc:729] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eaf9b43fcec4d849eb22385dd22f1a8, State: Initialized, Role: FOLLOWER
I20260929 16:07:48.640749 2926 raft_consensus.cc:374] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:48.640861 2926 raft_consensus.cc:729] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18eb6e2c53b343fe8b3054da835182d0, State: Initialized, Role: FOLLOWER
I20260929 16:07:48.641543 2927 consensus_queue.cc:260] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } }
I20260929 16:07:48.641600 2926 consensus_queue.cc:260] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } }
I20260929 16:07:48.641484 2928 raft_consensus.cc:348] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } }
I20260929 16:07:48.641932 2928 raft_consensus.cc:374] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:48.642083 2928 raft_consensus.cc:729] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cbbbd02f9447dca873969bee33ca24, State: Initialized, Role: FOLLOWER
I20260929 16:07:48.642866 2928 consensus_queue.cc:260] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } }
I20260929 16:07:48.644044 2926 ts_tablet_manager.cc:1428] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260929 16:07:48.645625 2773 heartbeater.cc:503] Master 127.20.121.190:38247 was elected leader, sending a full tablet report...
I20260929 16:07:48.646178 2927 ts_tablet_manager.cc:1428] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: Time spent starting tablet: real 0.016s user 0.006s sys 0.009s
I20260929 16:07:48.648841 2928 ts_tablet_manager.cc:1428] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: Time spent starting tablet: real 0.018s user 0.007s sys 0.011s
I20260929 16:07:48.649204 2907 heartbeater.cc:503] Master 127.20.121.190:38247 was elected leader, sending a full tablet report...
W20260929 16:07:48.671217 2640 tablet.cc:2369] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:07:48.712201 2774 tablet.cc:2369] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:07:48.753777 2908 tablet.cc:2369] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:07:48.817739 2932 raft_consensus.cc:482] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:07:48.818141 2932 raft_consensus.cc:504] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } }
I20260929 16:07:48.819903 2932 leader_election.cc:288] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141), 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105)
I20260929 16:07:48.830541 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.820610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:56421 (local address 127.20.121.130:38141)
0929 16:07:48.820793 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:48.820800 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:48.821649 (+ 849us) server_negotiation.cc:407] Connection header received
0929 16:07:48.821866 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:48.821873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:48.822098 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:48.822195 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:48.823134 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.824411 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.827709 (+ 3298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.829194 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.829376 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.829462 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:48.830117 (+ 655us) server_negotiation.cc:299] Negotiation successful
0929 16:07:48.830257 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 16:07:48.830557 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.820521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:54259 (local address 127.20.121.131:43105)
0929 16:07:48.820740 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:48.820750 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:48.821714 (+ 964us) server_negotiation.cc:407] Connection header received
0929 16:07:48.821866 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:48.821873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:48.821983 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:48.822120 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:48.823134 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.824411 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.827708 (+ 3297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.829127 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.829420 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.829502 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:48.830101 (+ 599us) server_negotiation.cc:299] Negotiation successful
0929 16:07:48.830248 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 16:07:48.830726 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.820476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:38141 (local address 127.20.121.129:56421)
0929 16:07:48.821476 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:48.821498 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:48.821673 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:48.822211 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:48.822216 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:48.822264 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:48.822895 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.822911 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.824588 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.824594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.827488 (+ 2894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.827509 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.829500 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.829508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.829775 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.829900 (+ 125us) client_negotiation.cc:769] Sending connection context
0929 16:07:48.830069 (+ 169us) client_negotiation.cc:240] Negotiation successful
0929 16:07:48.830208 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"mutex_wait_us":325,"spinlock_wait_cycles":7040,"thread_start_us":465,"threads_started":1}
I20260929 16:07:48.831239 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.820316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:43105 (local address 127.20.121.129:54259)
0929 16:07:48.821476 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:48.821498 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:48.821716 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:48.822206 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:48.822213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:48.822255 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:48.822897 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.822911 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.824589 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.824594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.827488 (+ 2894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:48.827509 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.829511 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.829516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:48.829781 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.829900 (+ 119us) client_negotiation.cc:769] Sending connection context
0929 16:07:48.830035 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 16:07:48.830208 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":326,"spinlock_wait_cycles":41600,"thread_start_us":732,"threads_started":1}
I20260929 16:07:48.832152 2728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01e3e51659d9430881f77746fe654e66" candidate_uuid: "18eb6e2c53b343fe8b3054da835182d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" is_pre_election: true
I20260929 16:07:48.832782 2728 raft_consensus.cc:2454] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18eb6e2c53b343fe8b3054da835182d0 in term 0.
I20260929 16:07:48.833595 2528 leader_election.cc:302] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18eb6e2c53b343fe8b3054da835182d0, 6eaf9b43fcec4d849eb22385dd22f1a8; no voters:
I20260929 16:07:48.834160 2932 raft_consensus.cc:2781] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:07:48.834331 2932 raft_consensus.cc:482] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:07:48.834447 2932 raft_consensus.cc:3037] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:48.834272 2862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01e3e51659d9430881f77746fe654e66" candidate_uuid: "18eb6e2c53b343fe8b3054da835182d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62cbbbd02f9447dca873969bee33ca24" is_pre_election: true
I20260929 16:07:48.834910 2862 raft_consensus.cc:2454] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18eb6e2c53b343fe8b3054da835182d0 in term 0.
I20260929 16:07:48.836941 2932 raft_consensus.cc:504] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } }
I20260929 16:07:48.837550 2932 leader_election.cc:288] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 1 election: Requested vote from peers 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141), 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105)
I20260929 16:07:48.838217 2728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01e3e51659d9430881f77746fe654e66" candidate_uuid: "18eb6e2c53b343fe8b3054da835182d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8"
I20260929 16:07:48.838217 2862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01e3e51659d9430881f77746fe654e66" candidate_uuid: "18eb6e2c53b343fe8b3054da835182d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62cbbbd02f9447dca873969bee33ca24"
I20260929 16:07:48.838438 2728 raft_consensus.cc:3037] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:48.838467 2862 raft_consensus.cc:3037] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:48.841858 2862 raft_consensus.cc:2454] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18eb6e2c53b343fe8b3054da835182d0 in term 1.
I20260929 16:07:48.841871 2728 raft_consensus.cc:2454] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18eb6e2c53b343fe8b3054da835182d0 in term 1.
I20260929 16:07:48.842433 2527 leader_election.cc:302] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 18eb6e2c53b343fe8b3054da835182d0, 62cbbbd02f9447dca873969bee33ca24, 6eaf9b43fcec4d849eb22385dd22f1a8; no voters:
I20260929 16:07:48.842955 2932 raft_consensus.cc:2781] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:07:48.843505 2932 raft_consensus.cc:686] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 1 LEADER]: Becoming Leader. State: Replica: 18eb6e2c53b343fe8b3054da835182d0, State: Running, Role: LEADER
I20260929 16:07:48.844063 2932 consensus_queue.cc:237] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } }
I20260929 16:07:48.849747 2448 catalog_manager.cc:5495] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18eb6e2c53b343fe8b3054da835182d0 (127.20.121.129). New cstate: current_term: 1 leader_uuid: "18eb6e2c53b343fe8b3054da835182d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } health_report { overall_health: HEALTHY } } }
I20260929 16:07:48.897710 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.891523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.20.121.129:45329)
0929 16:07:48.891704 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:48.891708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:48.891725 (+ 17us) server_negotiation.cc:407] Connection header received
0929 16:07:48.892010 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:48.892015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:48.892118 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:48.892248 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:07:48.892832 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.893590 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.895299 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.895726 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.895834 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.896501 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:07:48.897096 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:07:48.897226 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:48.897411 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 16:07:48.897491 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 16:07:48.906383 2862 raft_consensus.cc:1261] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24 [term 1 FOLLOWER]: Refusing update from remote peer 18eb6e2c53b343fe8b3054da835182d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:07:48.906385 2728 raft_consensus.cc:1261] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 1 FOLLOWER]: Refusing update from remote peer 18eb6e2c53b343fe8b3054da835182d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:07:48.907418 2937 consensus_queue.cc:1035] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:07:48.907742 2932 consensus_queue.cc:1035] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:07:48.919605 2940 mvcc.cc:205] Tried to move back new op lower bound from 7334699290230263808 to 7334699289989414912. Current Snapshot: MvccSnapshot[applied={T|T < 7334699290230263808}]
I20260929 16:07:48.935364 2448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:07:48.935653 2448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:07:48.939630 2448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Sending DeleteTablet for 3 replicas of tablet 01e3e51659d9430881f77746fe654e66
I20260929 16:07:48.940982 2708 tablet_service.cc:1514] Processing DeleteTablet for tablet 01e3e51659d9430881f77746fe654e66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:07:48 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:55442
I20260929 16:07:48.941123 2574 tablet_service.cc:1514] Processing DeleteTablet for tablet 01e3e51659d9430881f77746fe654e66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:07:48 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:59814
I20260929 16:07:48.941143 2842 tablet_service.cc:1514] Processing DeleteTablet for tablet 01e3e51659d9430881f77746fe654e66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:07:48 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:48696
I20260929 16:07:48.941991 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 2511
I20260929 16:07:48.965903 2952 tablet_replica.cc:330] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: stopping tablet replica
I20260929 16:07:48.968858 2952 raft_consensus.cc:2229] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:07:48.969539 2952 raft_consensus.cc:2258] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:07:48.972910 2952 ts_tablet_manager.cc:1905] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 16:07:48 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:07:48.981726 2951 tablet_replica.cc:330] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: stopping tablet replica
I20260929 16:07:48.981959 2950 tablet_replica.cc:330] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: stopping tablet replica
I20260929 16:07:48.982368 2950 raft_consensus.cc:2229] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:07:48.982357 2951 raft_consensus.cc:2229] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:07:48.982941 2951 raft_consensus.cc:2258] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:07:48.983160 2952 ts_tablet_manager.cc:1918] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:07:48.983333 2952 log.cc:1196] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/01e3e51659d9430881f77746fe654e66
I20260929 16:07:48.983868 2952 ts_tablet_manager.cc:1939] T 01e3e51659d9430881f77746fe654e66 P 62cbbbd02f9447dca873969bee33ca24: Deleting consensus metadata
I20260929 16:07:48.984078 2950 raft_consensus.cc:2258] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:07:48.985844 2951 ts_tablet_manager.cc:1905] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:07:48.985833 2434 catalog_manager.cc:4841] TS 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105): tablet 01e3e51659d9430881f77746fe654e66 (table test-table [id=a8a9c8b01aba4b848df9cf9f95a06170]) successfully deleted
I20260929 16:07:48.989833 2950 ts_tablet_manager.cc:1905] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:07:48.993630 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:48.943357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59824 (local address 127.20.121.129:45329)
0929 16:07:48.943611 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:48.943620 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:48.965239 (+ 21619us) server_negotiation.cc:407] Connection header received
0929 16:07:48.966359 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:48.966365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:48.966473 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:48.966556 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:48.970218 (+ 3662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.971424 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.976126 (+ 4702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:48.976865 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:48.976973 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:48.985200 (+ 8227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:48.985215 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:48.985233 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:48.985297 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:48.989728 (+ 4431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:48.990850 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:48.990859 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:48.990863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:48.991062 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:48.991546 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:48.991551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:48.991555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:48.992025 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:48.992362 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:48.992988 (+ 626us) server_negotiation.cc:299] Negotiation successful
0929 16:07:48.993250 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260929 16:07:48.995617 2634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:48.997210 2951 ts_tablet_manager.cc:1918] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:07:48.997416 2951 log.cc:1196] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/01e3e51659d9430881f77746fe654e66
I20260929 16:07:48.998113 2951 ts_tablet_manager.cc:1939] T 01e3e51659d9430881f77746fe654e66 P 6eaf9b43fcec4d849eb22385dd22f1a8: Deleting consensus metadata
I20260929 16:07:49.000200 2435 catalog_manager.cc:4841] TS 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141): tablet 01e3e51659d9430881f77746fe654e66 (table test-table [id=a8a9c8b01aba4b848df9cf9f95a06170]) successfully deleted
I20260929 16:07:49.102615 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 2511
I20260929 16:07:49.123198 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 2645
W20260929 16:07:49.123317 2435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.121.129:45329 (error 108)
I20260929 16:07:49.125677 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:49.124852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59836)
0929 16:07:49.125349 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:49.125481 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":266,"threads_started":1}
W20260929 16:07:49.126061 2435 catalog_manager.cc:4584] TS 18eb6e2c53b343fe8b3054da835182d0 (127.20.121.129:45329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 01e3e51659d9430881f77746fe654e66: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Sep 29 16:07:49 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:07:49.140606 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:49.124084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55450 (local address 127.20.121.130:38141)
0929 16:07:49.124569 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:49.124574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:49.124589 (+ 15us) server_negotiation.cc:407] Connection header received
0929 16:07:49.124645 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:49.124649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:49.124717 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:49.124806 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:49.126605 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:49.127497 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:49.129075 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:49.129546 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:49.129682 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:49.134558 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:49.134577 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:49.134594 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:49.134638 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:49.137479 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:49.138546 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:49.138554 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:49.138557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:49.138694 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:49.139027 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:49.139030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:49.139032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:49.139420 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:49.139700 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:49.140020 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 16:07:49.140209 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377}
I20260929 16:07:49.141373 2768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:49.180052 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:49.179455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59844)
0929 16:07:49.179741 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:49.179866 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260929 16:07:49.181169 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:49.180764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59850)
0929 16:07:49.180920 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:49.181007 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 16:07:49.235589 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 2645
I20260929 16:07:49.241986 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:49.241513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59866)
0929 16:07:49.241709 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:49.241816 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 16:07:49.243268 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:49.242833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59880)
0929 16:07:49.243000 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:49.243087 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 16:07:49.253587 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 2779
Sep 29 16:07:49 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:07:49.270704 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:49.254533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48698 (local address 127.20.121.131:43105)
0929 16:07:49.254739 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:49.254744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:49.254789 (+ 45us) server_negotiation.cc:407] Connection header received
0929 16:07:49.254922 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:49.254926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:49.255019 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:49.255195 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:49.256494 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:49.257376 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:49.258969 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:49.259396 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:49.259530 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:49.264332 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:49.264350 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:49.264366 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:49.264408 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:49.267464 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:49.268268 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:49.268276 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:49.268278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:49.268439 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:49.269054 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:49.269058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:49.269060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:49.269456 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:49.269791 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:49.270240 (+ 449us) server_negotiation.cc:299] Negotiation successful
0929 16:07:49.270419 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 16:07:49.271512 2902 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:49.323683 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:49.323157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59890)
0929 16:07:49.323410 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:49.323519 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260929 16:07:49.324961 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:49.324468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59906)
0929 16:07:49.324687 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:49.324764 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260929 16:07:49.368273 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 2779
I20260929 16:07:49.385603 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 2418
I20260929 16:07:49.387135 2480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:49.502694 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:49.501987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59908)
0929 16:07:49.502362 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:49.502493 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260929 16:07:49.503010 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 2418
I20260929 16:07:49.525347 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38247
--webserver_interface=127.20.121.190
--webserver_port=32791
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:38247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:07:49.871927 2960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:49.872298 2960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:49.872457 2960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:49.880580 2960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:07:49.880769 2960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:49.880897 2960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:49.881006 2960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:07:49.881101 2960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:07:49.892743 2960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:38247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=32791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:49.894253 2960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:49.896148 2960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:49.922135 2966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:49.922022 2969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:49.922081 2967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:49.922802 2960 server_base.cc:1034] running on GCE node
I20260929 16:07:49.923898 2960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:49.925776 2960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:49.927065 2960 hybrid_clock.cc:648] HybridClock initialized: now 1790698069927025 us; error 56 us; skew 500 ppm
I20260929 16:07:49.927542 2960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:49 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698069, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:49.931479 2960 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:07:49.933452 2960 webserver.cc:460] Webserver started at http://127.20.121.190:32791/ using document root <none> and password file <none>
I20260929 16:07:49.934376 2960 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:49.934556 2960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:49.941319 2960 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.005s
I20260929 16:07:49.944725 2976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:49.946341 2960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:07:49.946563 2960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "a133be94e2924db3b07f69b2c6336df1"
format_stamp: "Formatted at 2026-09-29 16:07:46 on dist-test-slave-x07d"
server_key: "1217aa5dfe0420bc6e97ab0e94e87290"
server_key_iv: "7aac5ec5ecb873c553a12a6f77ea63de"
server_key_version: "encryptionkey@0"
I20260929 16:07:49.947752 2960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:49.975330 2960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:49.980069 2960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:49.980369 2960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:50.001149 2960 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:38247
I20260929 16:07:50.001169 3027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:38247 every 8 connection(s)
I20260929 16:07:50.002920 2960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:07:50.010725 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 2960
I20260929 16:07:50.010831 3028 sys_catalog.cc:263] Verifying existing consensus state
I20260929 16:07:50.011757 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:45329
--local_ip_for_outbound_sockets=127.20.121.129
--tserver_master_addrs=127.20.121.190:38247
--webserver_port=33953
--webserver_interface=127.20.121.129
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:07:50.016502 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Bootstrap starting.
I20260929 16:07:50.038341 3028 log.cc:824] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:50.051183 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 16:07:50.051810 3028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Bootstrap complete.
I20260929 16:07:50.061678 3028 raft_consensus.cc:348] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:50.062637 3028 raft_consensus.cc:729] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a133be94e2924db3b07f69b2c6336df1, State: Initialized, Role: FOLLOWER
I20260929 16:07:50.063405 3028 consensus_queue.cc:260] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:50.063643 3028 raft_consensus.cc:388] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:07:50.063791 3028 raft_consensus.cc:482] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:07:50.064046 3028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 1 FOLLOWER]: Advancing to term 2
I20260929 16:07:50.068096 3028 raft_consensus.cc:504] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:50.068645 3028 leader_election.cc:302] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a133be94e2924db3b07f69b2c6336df1; no voters:
I20260929 16:07:50.069293 3028 leader_election.cc:288] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 16:07:50.069444 3032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 16:07:50.070902 3032 raft_consensus.cc:686] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 2 LEADER]: Becoming Leader. State: Replica: a133be94e2924db3b07f69b2c6336df1, State: Running, Role: LEADER
I20260929 16:07:50.071504 3032 consensus_queue.cc:237] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:50.071846 3028 sys_catalog.cc:564] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:07:50.074395 3033 sys_catalog.cc:455] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a133be94e2924db3b07f69b2c6336df1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } } }
I20260929 16:07:50.074867 3033 sys_catalog.cc:458] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:50.078826 3034 sys_catalog.cc:455] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a133be94e2924db3b07f69b2c6336df1. Latest consensus state: current_term: 2 leader_uuid: "a133be94e2924db3b07f69b2c6336df1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } } }
I20260929 16:07:50.079229 3034 sys_catalog.cc:458] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:50.080895 3038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:07:50.090473 3038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:07:50.092295 3038 catalog_manager.cc:1259] Loaded cluster ID: 25a803d4827c4190a6383d9b85019e59
I20260929 16:07:50.092509 3038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:07:50.096201 3038 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 16:07:50.118039 3038 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 16:07:50.123800 3049 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 16:07:50.128579 3038 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:07:50.130631 3038 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Loaded TSK: 0
I20260929 16:07:50.132436 3038 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 16:07:50.393021 3030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:50.393381 3030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:50.393538 3030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:50.401300 3030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:50.401482 3030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:50.401711 3030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:07:50.412276 3030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:45329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=33953
--tserver_master_addrs=127.20.121.190:38247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:50.413714 3030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:50.415123 3030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:50.430778 3055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:50.430822 3056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:50.430747 3058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:50.431979 3030 server_base.cc:1034] running on GCE node
I20260929 16:07:50.432816 3030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:50.433879 3030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:50.435168 3030 hybrid_clock.cc:648] HybridClock initialized: now 1790698070435094 us; error 58 us; skew 500 ppm
I20260929 16:07:50.435534 3030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:50 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698070, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:50.439612 3030 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:07:50.441689 3030 webserver.cc:460] Webserver started at http://127.20.121.129:33953/ using document root <none> and password file <none>
I20260929 16:07:50.442627 3030 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:50.442806 3030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:50.449314 3030 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 16:07:50.453244 3065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:50.454978 3030 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:07:50.455183 3030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "18eb6e2c53b343fe8b3054da835182d0"
format_stamp: "Formatted at 2026-09-29 16:07:47 on dist-test-slave-x07d"
server_key: "204228a45c98fa2fdd2b842602ccdfc1"
server_key_iv: "289663eaf442825eda69925ca48320d3"
server_key_version: "encryptionkey@0"
I20260929 16:07:50.456326 3030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:50.477531 3030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:50.482847 3030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:50.483201 3030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:50.484498 3030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:50.486654 3072 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260929 16:07:50.494558 3072 ts_tablet_manager.cc:1852] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260929 16:07:50.496083 3072 ts_tablet_manager.cc:1905] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:07:50.505054 3072 ts_tablet_manager.cc:1918] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:07:50.505300 3072 log.cc:1196] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/01e3e51659d9430881f77746fe654e66
I20260929 16:07:50.505821 3072 ts_tablet_manager.cc:1939] T 01e3e51659d9430881f77746fe654e66 P 18eb6e2c53b343fe8b3054da835182d0: Deleting consensus metadata
I20260929 16:07:50.506424 3030 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260929 16:07:50.506578 3030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.020s user 0.000s sys 0.000s
I20260929 16:07:50.506749 3030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:50.506842 3030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:50.543457 3030 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:45329
I20260929 16:07:50.543553 3178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:45329 every 8 connection(s)
I20260929 16:07:50.544818 3030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:07:50.550882 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 3030
I20260929 16:07:50.551697 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:38141
--local_ip_for_outbound_sockets=127.20.121.130
--tserver_master_addrs=127.20.121.190:38247
--webserver_port=38903
--webserver_interface=127.20.121.130
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:07:50 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698070, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:50.575884 3182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:50.548117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:43327)
0929 16:07:50.548982 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:50.549033 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:50.551630 (+ 2597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:50.553620 (+ 1990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:50.553635 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:50.554599 (+ 964us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:50.555482 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:50.555514 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:50.557487 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:50.557495 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:50.559598 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:50.559614 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:50.561222 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:50.561228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:50.561535 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:50.562345 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:50.562373 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:50.564915 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:50.570395 (+ 5480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:50.570409 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:50.570440 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:50.571032 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:50.571754 (+ 722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:50.571759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:50.571762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:50.572219 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:50.573519 (+ 1300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:50.573527 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:50.573990 (+ 463us) client_negotiation.cc:769] Sending connection context
0929 16:07:50.574333 (+ 343us) client_negotiation.cc:240] Negotiation successful
0929 16:07:50.574803 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":329,"threads_started":1}
I20260929 16:07:50.576647 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:50.547851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:43327 (local address 127.20.121.190:38247)
0929 16:07:50.548612 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:50.548644 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:50.549193 (+ 549us) server_negotiation.cc:407] Connection header received
0929 16:07:50.552470 (+ 3277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:50.552484 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:50.553059 (+ 575us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:50.553390 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:50.555793 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:50.557278 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:50.559774 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:50.560926 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:50.561139 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:50.565333 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:50.565348 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:50.565373 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:50.565438 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:50.570112 (+ 4674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:50.571248 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:50.571263 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:50.571274 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:50.571567 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:50.572391 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:50.572397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:50.572402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:50.573270 (+ 868us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:50.573521 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:50.574661 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0929 16:07:50.575204 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":311,"threads_started":1}
I20260929 16:07:50.580086 3179 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38247
I20260929 16:07:50.580492 3179 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:50.581365 3179 heartbeater.cc:511] Master 127.20.121.190:38247 requested a full tablet report, sending...
I20260929 16:07:50.583952 2993 ts_manager.cc:194] Registered new tserver with Master: 18eb6e2c53b343fe8b3054da835182d0 (127.20.121.129:45329)
I20260929 16:07:50.585660 2993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:43327
W20260929 16:07:50.905264 3183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:50.905583 3183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:50.905777 3183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:50.913481 3183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:50.913622 3183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:50.913878 3183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:07:50.924304 3183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:38141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=38903
--tserver_master_addrs=127.20.121.190:38247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:50.925738 3183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:50.927197 3183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:50.941933 3188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:50.942133 3189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:50.942888 3191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:50.943724 3183 server_base.cc:1034] running on GCE node
I20260929 16:07:50.944424 3183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:50.945571 3183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:50.946815 3183 hybrid_clock.cc:648] HybridClock initialized: now 1790698070946739 us; error 65 us; skew 500 ppm
I20260929 16:07:50.947213 3183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:50 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698070, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:50.950640 3183 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:07:50.952273 3183 webserver.cc:460] Webserver started at http://127.20.121.130:38903/ using document root <none> and password file <none>
I20260929 16:07:50.953043 3183 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:50.953203 3183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:50.958776 3183 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 16:07:50.961762 3198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:50.963531 3183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 16:07:50.963729 3183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "6eaf9b43fcec4d849eb22385dd22f1a8"
format_stamp: "Formatted at 2026-09-29 16:07:47 on dist-test-slave-x07d"
server_key: "779435239a1a9e3f56ca6c892fd15b69"
server_key_iv: "dc980135bc3a5f29be31df9655eaf1bc"
server_key_version: "encryptionkey@0"
I20260929 16:07:50.964718 3183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:50.983114 3183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:50.987886 3183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:50.988180 3183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:50.989298 3183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:50.990764 3183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:50.990852 3183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:50.990998 3183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:50.991073 3183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:51.029486 3183 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:38141
I20260929 16:07:51.029593 3310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:38141 every 8 connection(s)
I20260929 16:07:51.031529 3183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:07:51.040278 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 3183
I20260929 16:07:51.040994 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:43105
--local_ip_for_outbound_sockets=127.20.121.131
--tserver_master_addrs=127.20.121.190:38247
--webserver_port=38357
--webserver_interface=127.20.121.131
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:07:51 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698070, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:51.053256 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.034701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:54151 (local address 127.20.121.190:38247)
0929 16:07:51.034940 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.034945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.036303 (+ 1358us) server_negotiation.cc:407] Connection header received
0929 16:07:51.038231 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.038237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.038364 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.038482 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:51.040052 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.040839 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.042654 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.043075 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.043215 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.046872 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:51.046877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:51.046880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:51.046916 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:51.049775 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:51.050679 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:51.050692 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:51.050694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:51.050830 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:51.051339 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:51.051346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:51.051353 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:51.051636 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:51.051839 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.052623 (+ 784us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.052890 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260929 16:07:51.054342 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.035312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:54151)
0929 16:07:51.036099 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:51.036138 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:51.037909 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:51.038727 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:51.038735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:51.039271 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:51.039881 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.039898 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.040993 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.041003 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.042506 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.042518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.043334 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.043341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.043568 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.044286 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:51.044313 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:51.046633 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:51.049961 (+ 3328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:51.049971 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:51.049990 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:51.050538 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:51.051004 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:51.051010 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:51.051014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:51.051207 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:51.051789 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:51.051798 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:51.052235 (+ 437us) client_negotiation.cc:769] Sending connection context
0929 16:07:51.052512 (+ 277us) client_negotiation.cc:240] Negotiation successful
0929 16:07:51.053090 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":325,"threads_started":1}
I20260929 16:07:51.055949 3311 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38247
I20260929 16:07:51.056286 3311 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:51.057120 3311 heartbeater.cc:511] Master 127.20.121.190:38247 requested a full tablet report, sending...
I20260929 16:07:51.058830 2993 ts_manager.cc:194] Registered new tserver with Master: 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141)
I20260929 16:07:51.060249 2993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:54151
W20260929 16:07:51.392903 3314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:51.393213 3314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:51.393313 3314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:51.401189 3314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:51.401321 3314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:51.401438 3314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:07:51.411566 3314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:43105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=38357
--tserver_master_addrs=127.20.121.190:38247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:51.412853 3314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:51.414223 3314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:51.427886 3319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:51.427832 3320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:51.428475 3322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:51.428656 3314 server_base.cc:1034] running on GCE node
I20260929 16:07:51.429358 3314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:51.430532 3314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:51.431813 3314 hybrid_clock.cc:648] HybridClock initialized: now 1790698071431726 us; error 60 us; skew 500 ppm
I20260929 16:07:51.432202 3314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:51 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698071, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:51.436077 3314 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:07:51.438261 3314 webserver.cc:460] Webserver started at http://127.20.121.131:38357/ using document root <none> and password file <none>
I20260929 16:07:51.439059 3314 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:51.439224 3314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:51.444859 3314 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260929 16:07:51.447911 3329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:51.449476 3314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:07:51.449673 3314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "62cbbbd02f9447dca873969bee33ca24"
format_stamp: "Formatted at 2026-09-29 16:07:48 on dist-test-slave-x07d"
server_key: "c6ce0e1167e46f3953e2934939af69d9"
server_key_iv: "7342659e2deb3325ba505eb583101dd4"
server_key_version: "encryptionkey@0"
I20260929 16:07:51.450767 3314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:51.483999 3314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:51.489086 3314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:51.489460 3314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:51.490723 3314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:51.492282 3314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:51.492426 3314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:51.492570 3314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:51.492662 3314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:51.529062 3314 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:43105
I20260929 16:07:51.529163 3441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:43105 every 8 connection(s)
I20260929 16:07:51.530695 3314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:07:51.536638 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 3314
Sep 29 16:07:51 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698071, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:07:51.556339 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.533633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:36545 (local address 127.20.121.190:38247)
0929 16:07:51.533845 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.533849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.535246 (+ 1397us) server_negotiation.cc:407] Connection header received
0929 16:07:51.537373 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.537378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.537483 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.537597 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:51.540538 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.541581 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.544119 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.544670 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.544824 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.549091 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:51.549096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:51.549099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:51.549131 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:51.552147 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:51.553271 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:51.553277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:51.553281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:51.553450 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:51.554111 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:51.554115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:51.554117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:51.554495 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:51.554670 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.555691 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.555952 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":76}
I20260929 16:07:51.557374 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.534245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:36545)
0929 16:07:51.535039 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:51.535100 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:51.537046 (+ 1946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:51.538091 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:51.538110 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:51.539143 (+ 1033us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:51.540266 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.540303 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.541761 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.541769 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.543953 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.543969 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.544954 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.544964 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.545299 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.546182 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:51.546216 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:51.548843 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:51.552353 (+ 3510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:51.552390 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:51.552411 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:51.553110 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:51.553597 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:51.553608 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:51.553611 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:51.553937 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:51.554647 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:51.554656 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:51.555047 (+ 391us) client_negotiation.cc:769] Sending connection context
0929 16:07:51.555484 (+ 437us) client_negotiation.cc:240] Negotiation successful
0929 16:07:51.556094 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":306,"threads_started":1}
I20260929 16:07:51.558894 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.537606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.20.121.190:38247)
0929 16:07:51.538158 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.538162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.541102 (+ 2940us) server_negotiation.cc:407] Connection header received
0929 16:07:51.541221 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.541226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.541356 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.541504 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:51.543706 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.544976 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.547031 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.547721 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.547855 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.551947 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:51.551953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:51.551957 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:51.552003 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:51.555073 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:51.555995 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:51.556000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:51.556005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:51.556135 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:51.556763 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:51.556766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:51.556768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:51.557057 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:51.557354 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.558218 (+ 864us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.558486 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":457,"thread_start_us":298,"threads_started":1}
I20260929 16:07:51.559297 3442 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38247
I20260929 16:07:51.559650 3442 heartbeater.cc:464] Registering TS with master...
I20260929 16:07:51.560608 3442 heartbeater.cc:511] Master 127.20.121.190:38247 requested a full tablet report, sending...
I20260929 16:07:51.562482 2993 ts_manager.cc:194] Registered new tserver with Master: 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105)
I20260929 16:07:51.563381 2993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:36545
I20260929 16:07:51.574730 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:07:51.582345 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.575945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41704 (local address 127.20.121.190:38247)
0929 16:07:51.576192 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.576199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.576322 (+ 123us) server_negotiation.cc:407] Connection header received
0929 16:07:51.576599 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.576603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.576745 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.576908 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:07:51.577373 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.578173 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.579781 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.580297 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.580452 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.580962 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:07:51.581622 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:07:51.581801 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.581892 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.581985 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260929 16:07:51.589358 3179 heartbeater.cc:503] Master 127.20.121.190:38247 was elected leader, sending a full tablet report...
I20260929 16:07:51.583577 2993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:07:51.591861 2993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:07:51.627725 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.604846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55452 (local address 127.20.121.130:38141)
0929 16:07:51.615924 (+ 11078us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.615934 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.617187 (+ 1253us) server_negotiation.cc:407] Connection header received
0929 16:07:51.617310 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.617322 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.617600 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.617829 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:51.619222 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.621208 (+ 1986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.624799 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.626476 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.626683 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.626877 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.627226 (+ 349us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.627402 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10929,"thread_start_us":10850,"threads_started":1}
I20260929 16:07:51.627784 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.604584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:38141 (local address 127.0.0.1:55452)
0929 16:07:51.616900 (+ 12316us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:51.616923 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:51.617051 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:51.618224 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:51.618237 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:51.618280 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:51.618908 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.618927 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.621401 (+ 2474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.621407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.624563 (+ 3156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.624581 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.626717 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.626723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.626949 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.627003 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 16:07:51.627155 (+ 152us) client_negotiation.cc:240] Negotiation successful
0929 16:07:51.627403 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11598,"thread_start_us":11529,"threads_started":1}
I20260929 16:07:51.630599 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.605890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48708 (local address 127.20.121.131:43105)
0929 16:07:51.615670 (+ 9780us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.615681 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.616654 (+ 973us) server_negotiation.cc:407] Connection header received
0929 16:07:51.616985 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.616998 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.617308 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.617514 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:51.620480 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.622785 (+ 2305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.627390 (+ 4605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.629119 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.629278 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.629476 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.630061 (+ 585us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.630234 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9572,"thread_start_us":9395,"threads_started":1}
I20260929 16:07:51.631816 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.605382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:43105 (local address 127.0.0.1:48708)
0929 16:07:51.616452 (+ 11070us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:51.616506 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:51.616753 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:51.619676 (+ 2923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:51.619684 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:51.619723 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:51.620212 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.620226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.623052 (+ 2826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.623060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.627161 (+ 4101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.627173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.629681 (+ 2508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.629687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.629823 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.629849 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 16:07:51.629970 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 16:07:51.631475 (+ 1505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10854,"thread_start_us":11044,"threads_started":1}
I20260929 16:07:51.632306 3246 tablet_service.cc:1467] Processing CreateTablet for tablet c4c06f9613654011994fdc656cac9d6f (DEFAULT_TABLE table=test-table [id=017782b321fe40ee8248d949243bf52c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:07:51.633944 3246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4c06f9613654011994fdc656cac9d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:51.634222 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.617469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59918 (local address 127.20.121.129:45329)
0929 16:07:51.625974 (+ 8505us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.625983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.626036 (+ 53us) server_negotiation.cc:407] Connection header received
0929 16:07:51.626366 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.626376 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.626650 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.626871 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:51.627785 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.629349 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.631877 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.633038 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.633241 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.633427 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.633646 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.633813 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8309,"thread_start_us":8134,"threads_started":1}
I20260929 16:07:51.634696 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.617127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59918)
0929 16:07:51.624790 (+ 7663us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:51.624817 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:51.626152 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:51.627126 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:51.627131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:51.627158 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:51.627565 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.627579 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.629493 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.629498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.631671 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.631690 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.633253 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.633259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.633429 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.633473 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 16:07:51.633584 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 16:07:51.633721 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7477,"thread_start_us":8142,"threads_started":1}
I20260929 16:07:51.636148 3377 tablet_service.cc:1467] Processing CreateTablet for tablet c4c06f9613654011994fdc656cac9d6f (DEFAULT_TABLE table=test-table [id=017782b321fe40ee8248d949243bf52c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:07:51.636879 3114 tablet_service.cc:1467] Processing CreateTablet for tablet c4c06f9613654011994fdc656cac9d6f (DEFAULT_TABLE table=test-table [id=017782b321fe40ee8248d949243bf52c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:07:51.637395 3377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4c06f9613654011994fdc656cac9d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:51.638180 3114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4c06f9613654011994fdc656cac9d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:07:51.650350 3459 tablet_bootstrap.cc:492] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: Bootstrap starting.
I20260929 16:07:51.652990 3458 tablet_bootstrap.cc:492] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Bootstrap starting.
I20260929 16:07:51.654827 3460 tablet_bootstrap.cc:492] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: Bootstrap starting.
I20260929 16:07:51.655529 3459 tablet_bootstrap.cc:654] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:51.656754 3459 log.cc:824] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:51.657013 3458 tablet_bootstrap.cc:654] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:51.658346 3458 log.cc:824] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:51.658444 3460 tablet_bootstrap.cc:654] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: Neither blocks nor log segments found. Creating new log.
I20260929 16:07:51.659670 3460 log.cc:824] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:51.660984 3459 tablet_bootstrap.cc:492] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: No bootstrap required, opened a new log
I20260929 16:07:51.661290 3459 ts_tablet_manager.cc:1397] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260929 16:07:51.661751 3458 tablet_bootstrap.cc:492] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: No bootstrap required, opened a new log
I20260929 16:07:51.662071 3458 ts_tablet_manager.cc:1397] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 16:07:51.663031 3460 tablet_bootstrap.cc:492] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: No bootstrap required, opened a new log
I20260929 16:07:51.663312 3460 ts_tablet_manager.cc:1397] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 16:07:51.668991 3459 raft_consensus.cc:348] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:51.669466 3459 raft_consensus.cc:374] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:51.669600 3459 raft_consensus.cc:729] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eaf9b43fcec4d849eb22385dd22f1a8, State: Initialized, Role: FOLLOWER
I20260929 16:07:51.670286 3459 consensus_queue.cc:260] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:51.672418 3311 heartbeater.cc:503] Master 127.20.121.190:38247 was elected leader, sending a full tablet report...
I20260929 16:07:51.673539 3459 ts_tablet_manager.cc:1428] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260929 16:07:51.674404 3460 raft_consensus.cc:348] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:51.674923 3460 raft_consensus.cc:374] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:51.675028 3460 raft_consensus.cc:729] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cbbbd02f9447dca873969bee33ca24, State: Initialized, Role: FOLLOWER
I20260929 16:07:51.674897 3458 raft_consensus.cc:348] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:51.675348 3458 raft_consensus.cc:374] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:07:51.675475 3458 raft_consensus.cc:729] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18eb6e2c53b343fe8b3054da835182d0, State: Initialized, Role: FOLLOWER
I20260929 16:07:51.675817 3460 consensus_queue.cc:260] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:51.676051 3458 consensus_queue.cc:260] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:51.679704 3458 ts_tablet_manager.cc:1428] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Time spent starting tablet: real 0.017s user 0.000s sys 0.013s
I20260929 16:07:51.680817 3442 heartbeater.cc:503] Master 127.20.121.190:38247 was elected leader, sending a full tablet report...
I20260929 16:07:51.681516 3460 ts_tablet_manager.cc:1428] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
W20260929 16:07:51.782598 3443 tablet.cc:2369] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:07:51.783592 3312 tablet.cc:2369] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:07:51.797429 3180 tablet.cc:2369] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:07:51.807205 3465 raft_consensus.cc:482] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:07:51.807528 3465 raft_consensus.cc:504] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:51.809331 3465 leader_election.cc:288] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141), 18eb6e2c53b343fe8b3054da835182d0 (127.20.121.129:45329)
I20260929 16:07:51.816975 3444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.809906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:38141 (local address 127.20.121.131:56027)
0929 16:07:51.810195 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:51.810214 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:51.810385 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:51.810939 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:51.810942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:51.810969 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:51.811136 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.811144 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.812557 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.812563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.814559 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.814572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.816144 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.816150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.816344 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.816386 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 16:07:51.816496 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 16:07:51.816659 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260929 16:07:51.817003 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.810052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:56027 (local address 127.20.121.130:38141)
0929 16:07:51.810311 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.810318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.810342 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:07:51.810572 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.810576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.810637 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.810743 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:51.811309 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.812406 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.814752 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.815974 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.816101 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.816160 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.816568 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.816693 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 16:07:51.818038 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.810616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:57359 (local address 127.20.121.129:45329)
0929 16:07:51.810841 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.810847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.811339 (+ 492us) server_negotiation.cc:407] Connection header received
0929 16:07:51.811503 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.811506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.811572 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.811642 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:07:51.812293 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.813385 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.815694 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.816891 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.817068 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.817150 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.817601 (+ 451us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.817755 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 16:07:51.818084 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.810416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.20.121.131:57359)
0929 16:07:51.811190 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:51.811233 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:51.811383 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:51.811700 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:51.811705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:51.811730 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:07:51.812078 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.812088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.813574 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.813578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.815491 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:51.815508 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.817145 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.817151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:51.817400 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.817448 (+ 48us) client_negotiation.cc:769] Sending connection context
0929 16:07:51.817564 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 16:07:51.817709 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":428,"threads_started":1}
I20260929 16:07:51.818511 3266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c06f9613654011994fdc656cac9d6f" candidate_uuid: "62cbbbd02f9447dca873969bee33ca24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" is_pre_election: true
I20260929 16:07:51.819118 3266 raft_consensus.cc:2454] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cbbbd02f9447dca873969bee33ca24 in term 0.
I20260929 16:07:51.819156 3134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c06f9613654011994fdc656cac9d6f" candidate_uuid: "62cbbbd02f9447dca873969bee33ca24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18eb6e2c53b343fe8b3054da835182d0" is_pre_election: true
I20260929 16:07:51.819700 3134 raft_consensus.cc:2454] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cbbbd02f9447dca873969bee33ca24 in term 0.
I20260929 16:07:51.819972 3331 leader_election.cc:302] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cbbbd02f9447dca873969bee33ca24, 6eaf9b43fcec4d849eb22385dd22f1a8; no voters:
I20260929 16:07:51.820436 3465 raft_consensus.cc:2781] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:07:51.820595 3465 raft_consensus.cc:482] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:07:51.820710 3465 raft_consensus.cc:3037] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:51.823231 3465 raft_consensus.cc:504] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:51.823889 3465 leader_election.cc:288] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [CANDIDATE]: Term 1 election: Requested vote from peers 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141), 18eb6e2c53b343fe8b3054da835182d0 (127.20.121.129:45329)
I20260929 16:07:51.824712 3266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c06f9613654011994fdc656cac9d6f" candidate_uuid: "62cbbbd02f9447dca873969bee33ca24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8"
I20260929 16:07:51.824734 3134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4c06f9613654011994fdc656cac9d6f" candidate_uuid: "62cbbbd02f9447dca873969bee33ca24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18eb6e2c53b343fe8b3054da835182d0"
I20260929 16:07:51.824967 3266 raft_consensus.cc:3037] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:51.825052 3134 raft_consensus.cc:3037] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:07:51.828506 3134 raft_consensus.cc:2454] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cbbbd02f9447dca873969bee33ca24 in term 1.
I20260929 16:07:51.828506 3266 raft_consensus.cc:2454] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cbbbd02f9447dca873969bee33ca24 in term 1.
I20260929 16:07:51.829087 3331 leader_election.cc:302] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cbbbd02f9447dca873969bee33ca24, 6eaf9b43fcec4d849eb22385dd22f1a8; no voters:
I20260929 16:07:51.829581 3465 raft_consensus.cc:2781] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:07:51.830202 3465 raft_consensus.cc:686] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 1 LEADER]: Becoming Leader. State: Replica: 62cbbbd02f9447dca873969bee33ca24, State: Running, Role: LEADER
I20260929 16:07:51.830757 3465 consensus_queue.cc:237] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:51.836030 2992 catalog_manager.cc:5495] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "62cbbbd02f9447dca873969bee33ca24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } health_report { overall_health: HEALTHY } } }
I20260929 16:07:51.860963 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.855588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48716 (local address 127.20.121.131:43105)
0929 16:07:51.855775 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.855780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.855798 (+ 18us) server_negotiation.cc:407] Connection header received
0929 16:07:51.855997 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.856001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.856081 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.856213 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:07:51.856589 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.857318 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.858761 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.859161 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.859258 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.859854 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:07:51.860414 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:07:51.860526 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.860652 (+ 126us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.860720 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 16:07:51.869576 3266 raft_consensus.cc:1261] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 1 FOLLOWER]: Refusing update from remote peer 62cbbbd02f9447dca873969bee33ca24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:07:51.869628 3134 raft_consensus.cc:1261] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Refusing update from remote peer 62cbbbd02f9447dca873969bee33ca24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:07:51.870711 3468 consensus_queue.cc:1035] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:07:51.871114 3465 consensus_queue.cc:1035] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:07:51.904824 2993 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:07:51.905192 2993 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:07:51.909518 2993 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Sending DeleteTablet for 3 replicas of tablet c4c06f9613654011994fdc656cac9d6f
I20260929 16:07:51.911151 3114 tablet_service.cc:1514] Processing DeleteTablet for tablet c4c06f9613654011994fdc656cac9d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:07:51 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:59918
I20260929 16:07:51.911101 3377 tablet_service.cc:1514] Processing DeleteTablet for tablet c4c06f9613654011994fdc656cac9d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:07:51 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:48708
I20260929 16:07:51.911521 3246 tablet_service.cc:1514] Processing DeleteTablet for tablet c4c06f9613654011994fdc656cac9d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:07:51 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:55452
I20260929 16:07:51.912449 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 3030
I20260929 16:07:51.947525 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:51.913542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.20.121.129:45329)
0929 16:07:51.914418 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:51.914428 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:51.914455 (+ 27us) server_negotiation.cc:407] Connection header received
0929 16:07:51.914533 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:51.914541 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:51.914645 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:51.914763 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:51.917481 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.919136 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.922056 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:51.923072 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:51.923237 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:51.933402 (+ 10165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:51.933433 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:51.933462 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:51.933550 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:51.939434 (+ 5884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:51.940959 (+ 1525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:51.940977 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:51.940984 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:51.941258 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:51.943084 (+ 1826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:51.943093 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:51.943099 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:51.943809 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:51.944222 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:51.946562 (+ 2340us) server_negotiation.cc:299] Negotiation successful
0929 16:07:51.946917 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680}
I20260929 16:07:51.950364 3174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:51.955488 3483 tablet_replica.cc:330] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: stopping tablet replica
I20260929 16:07:51.955765 3482 tablet_replica.cc:330] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: stopping tablet replica
I20260929 16:07:51.956050 3483 raft_consensus.cc:2229] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:07:51.956314 3482 raft_consensus.cc:2229] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:07:51.956676 3483 raft_consensus.cc:2258] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:07:51.957293 3482 raft_consensus.cc:2258] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:07:51.958986 3483 ts_tablet_manager.cc:1905] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:07:51.959671 3482 ts_tablet_manager.cc:1905] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:07:51.967267 3483 ts_tablet_manager.cc:1918] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:07:51.967532 3483 log.cc:1196] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/c4c06f9613654011994fdc656cac9d6f
I20260929 16:07:51.967528 3482 ts_tablet_manager.cc:1918] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:07:51.967720 3482 log.cc:1196] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/c4c06f9613654011994fdc656cac9d6f
I20260929 16:07:51.968168 3483 ts_tablet_manager.cc:1939] T c4c06f9613654011994fdc656cac9d6f P 6eaf9b43fcec4d849eb22385dd22f1a8: Deleting consensus metadata
I20260929 16:07:51.968482 3482 ts_tablet_manager.cc:1939] T c4c06f9613654011994fdc656cac9d6f P 62cbbbd02f9447dca873969bee33ca24: Deleting consensus metadata
I20260929 16:07:51.970701 2977 catalog_manager.cc:4841] TS 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105): tablet c4c06f9613654011994fdc656cac9d6f (table test-table [id=017782b321fe40ee8248d949243bf52c]) successfully deleted
I20260929 16:07:51.971107 2978 catalog_manager.cc:4841] TS 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141): tablet c4c06f9613654011994fdc656cac9d6f (table test-table [id=017782b321fe40ee8248d949243bf52c]) successfully deleted
I20260929 16:07:52.067760 3481 tablet_replica.cc:330] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: stopping tablet replica
I20260929 16:07:52.068311 3481 raft_consensus.cc:2229] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:07:52.068725 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 3030
I20260929 16:07:52.086670 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 3183
W20260929 16:07:52.086694 2978 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.121.129:45329 (error 108)
I20260929 16:07:52.088502 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:52.087938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59936)
0929 16:07:52.088182 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:52.088309 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260929 16:07:52.088780 2978 catalog_manager.cc:4584] TS 18eb6e2c53b343fe8b3054da835182d0 (127.20.121.129:45329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c4c06f9613654011994fdc656cac9d6f: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
I20260929 16:07:52.104239 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:52.087933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55460 (local address 127.20.121.130:38141)
0929 16:07:52.088185 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:52.088192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:52.088214 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:07:52.088284 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:52.088290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:52.088369 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:52.088484 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:52.090474 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:52.091225 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:52.093291 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:52.093663 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:52.093758 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:52.096861 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:52.096881 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:52.096904 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:52.096963 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:52.101146 (+ 4183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:52.101914 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:52.101925 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:52.101929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:52.102134 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:52.102494 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:52.102500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:52.102504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:52.103032 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:52.103362 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:52.103553 (+ 191us) server_negotiation.cc:299] Negotiation successful
0929 16:07:52.103816 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260929 16:07:52.105062 3306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:52.142664 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:52.142000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59948)
0929 16:07:52.142280 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:52.142467 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260929 16:07:52.144068 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:52.143586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59958)
0929 16:07:52.143778 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:52.143891 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 16:07:52.201815 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 3183
I20260929 16:07:52.205018 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:52.204385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59970)
0929 16:07:52.204651 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:52.204779 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260929 16:07:52.206384 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:52.205884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59982)
0929 16:07:52.206101 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:52.206218 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260929 16:07:52.220182 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 3314
I20260929 16:07:52.238368 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:52.221060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48726 (local address 127.20.121.131:43105)
0929 16:07:52.221247 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:52.221251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:52.221271 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:07:52.221516 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:52.221521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:52.221596 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:52.221688 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:52.223789 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:52.224670 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:52.226863 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:52.227351 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:52.227480 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:52.231469 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:52.231480 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:52.231494 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:52.231532 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:52.234920 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:52.235967 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:52.235974 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:52.235977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:52.236155 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:52.236617 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:52.236621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:52.236623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:52.237087 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:52.237383 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:52.237750 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 16:07:52.237935 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 16:07:52.239210 3437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:52.287724 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:52.287118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:59998)
0929 16:07:52.287387 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:52.287509 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260929 16:07:52.288897 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:52.288454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:60006)
0929 16:07:52.288665 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:52.288752 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260929 16:07:52.366891 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 3314
I20260929 16:07:52.387231 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 2960
I20260929 16:07:52.388670 3023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:07:52.509262 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 2960
I20260929 16:07:52.509297 3456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:52.508801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45329 (local address 127.0.0.1:60018)
0929 16:07:52.509020 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:52.509117 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:45329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 16:07:52.532462 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38247
--webserver_interface=127.20.121.190
--webserver_port=32791
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:38247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:07:52.883035 3489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:52.883463 3489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:52.883622 3489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:52.891073 3489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:07:52.891238 3489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:52.891350 3489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:52.891487 3489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:07:52.891567 3489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:07:52.902829 3489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:38247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=32791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:52.904263 3489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:52.905598 3489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:52.918314 3497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:52.918332 3494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:52.918417 3495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:52.919175 3489 server_base.cc:1034] running on GCE node
I20260929 16:07:52.920068 3489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:52.921622 3489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:52.922843 3489 hybrid_clock.cc:648] HybridClock initialized: now 1790698072922790 us; error 50 us; skew 500 ppm
I20260929 16:07:52.923237 3489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:52 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698072, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:52.926755 3489 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:07:52.928547 3489 webserver.cc:460] Webserver started at http://127.20.121.190:32791/ using document root <none> and password file <none>
I20260929 16:07:52.929311 3489 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:52.929461 3489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:52.935497 3489 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 16:07:52.938726 3504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:52.940230 3489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:07:52.940438 3489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "a133be94e2924db3b07f69b2c6336df1"
format_stamp: "Formatted at 2026-09-29 16:07:46 on dist-test-slave-x07d"
server_key: "1217aa5dfe0420bc6e97ab0e94e87290"
server_key_iv: "7aac5ec5ecb873c553a12a6f77ea63de"
server_key_version: "encryptionkey@0"
I20260929 16:07:52.941498 3489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:52.956522 3489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:52.961184 3489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:52.961478 3489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:52.981143 3489 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:38247
I20260929 16:07:52.981168 3555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:38247 every 8 connection(s)
I20260929 16:07:52.982772 3489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:07:52.988003 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 3489
I20260929 16:07:52.988822 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:45329
--local_ip_for_outbound_sockets=127.20.121.129
--tserver_master_addrs=127.20.121.190:38247
--webserver_port=33953
--webserver_interface=127.20.121.129
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:07:52.990401 3556 sys_catalog.cc:263] Verifying existing consensus state
I20260929 16:07:52.993700 3556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Bootstrap starting.
I20260929 16:07:53.024206 3556 log.cc:824] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:53.047465 3556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 16:07:53.048205 3556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1: Bootstrap complete.
I20260929 16:07:53.063537 3556 raft_consensus.cc:348] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:53.064826 3556 raft_consensus.cc:729] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a133be94e2924db3b07f69b2c6336df1, State: Initialized, Role: FOLLOWER
I20260929 16:07:53.065568 3556 consensus_queue.cc:260] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:53.065858 3556 raft_consensus.cc:388] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:07:53.065980 3556 raft_consensus.cc:482] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:07:53.066196 3556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 2 FOLLOWER]: Advancing to term 3
I20260929 16:07:53.071889 3556 raft_consensus.cc:504] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:53.072515 3556 leader_election.cc:302] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a133be94e2924db3b07f69b2c6336df1; no voters:
I20260929 16:07:53.073235 3556 leader_election.cc:288] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 16:07:53.073410 3560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 16:07:53.074795 3560 raft_consensus.cc:686] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [term 3 LEADER]: Becoming Leader. State: Replica: a133be94e2924db3b07f69b2c6336df1, State: Running, Role: LEADER
I20260929 16:07:53.075412 3556 sys_catalog.cc:564] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:07:53.075392 3560 consensus_queue.cc:237] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } }
I20260929 16:07:53.077888 3562 sys_catalog.cc:455] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a133be94e2924db3b07f69b2c6336df1. Latest consensus state: current_term: 3 leader_uuid: "a133be94e2924db3b07f69b2c6336df1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } } }
I20260929 16:07:53.077920 3561 sys_catalog.cc:455] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a133be94e2924db3b07f69b2c6336df1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a133be94e2924db3b07f69b2c6336df1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38247 } } }
I20260929 16:07:53.078410 3561 sys_catalog.cc:458] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:53.078413 3562 sys_catalog.cc:458] T 00000000000000000000000000000000 P a133be94e2924db3b07f69b2c6336df1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:07:53.082818 3567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:07:53.092640 3567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:07:53.094381 3567 catalog_manager.cc:1259] Loaded cluster ID: 25a803d4827c4190a6383d9b85019e59
I20260929 16:07:53.094515 3567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 16:07:53.096750 3567 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8c1c30e9ad google::LogMessage::Fail() at ??:0
@ 0x7f8c1c31188a google::LogMessage::SendToLog() at ??:0
@ 0x7f8c1c30e4de google::LogMessage::Flush() at ??:0
@ 0x7f8c1c30fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260929 16:07:53.104936 3577 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f8c31d41512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f8c31d3e976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8c31dc6610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f8c31dc6439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8c1ef5a345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8c1ef5d91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f8c1ef5d799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8c1ef38eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8c222126db start_thread at ??:0
@ 0x7f8c1b1a861f clone at ??:0
W20260929 16:07:53.437187 3558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:53.437536 3558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:53.437644 3558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:53.446578 3558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:53.446705 3558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:53.446839 3558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:07:53.458071 3558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:45329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=33953
--tserver_master_addrs=127.20.121.190:38247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:53.459496 3558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:53.460876 3558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:53.476452 3597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:53.476544 3598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:53.476498 3600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:53.477865 3558 server_base.cc:1034] running on GCE node
I20260929 16:07:53.478737 3558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:53.479811 3558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:53.481047 3558 hybrid_clock.cc:648] HybridClock initialized: now 1790698073480972 us; error 63 us; skew 500 ppm
I20260929 16:07:53.481434 3558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:53 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698073, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:53.489022 3558 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:07:53.490743 3558 webserver.cc:460] Webserver started at http://127.20.121.129:33953/ using document root <none> and password file <none>
I20260929 16:07:53.491461 3558 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:53.491564 3558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:53.497159 3558 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 16:07:53.500315 3607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:53.501966 3558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:07:53.502219 3558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "18eb6e2c53b343fe8b3054da835182d0"
format_stamp: "Formatted at 2026-09-29 16:07:47 on dist-test-slave-x07d"
server_key: "204228a45c98fa2fdd2b842602ccdfc1"
server_key_iv: "289663eaf442825eda69925ca48320d3"
server_key_version: "encryptionkey@0"
I20260929 16:07:53.503244 3558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:53.518529 3558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:53.522742 3558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:53.522989 3558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:53.523937 3558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:53.525906 3614 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260929 16:07:53.532284 3558 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 16:07:53.532447 3558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260929 16:07:53.532599 3558 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 16:07:53.535907 3558 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 16:07:53.536022 3558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260929 16:07:53.536403 3614 tablet_bootstrap.cc:492] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Bootstrap starting.
I20260929 16:07:53.567672 3614 log.cc:824] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Log is configured to *not* fsync() on all Append() calls
I20260929 16:07:53.578460 3558 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:45329
I20260929 16:07:53.578480 3721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:45329 every 8 connection(s)
I20260929 16:07:53.579851 3558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:07:53.582350 3614 tablet_bootstrap.cc:492] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 16:07:53.582957 3614 tablet_bootstrap.cc:492] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Bootstrap complete.
I20260929 16:07:53.584821 3614 ts_tablet_manager.cc:1397] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Time spent bootstrapping tablet: real 0.049s user 0.034s sys 0.011s
I20260929 16:07:53.585585 3724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:53.582986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:51241)
0929 16:07:53.583942 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:53.584288 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":358,"threads_started":1}
W20260929 16:07:53.586437 3722 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:38247 (0 consecutive failures): Network error: Failed to ping master at 127.20.121.190:38247: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
I20260929 16:07:53.587577 3724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:53.587074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:44049)
0929 16:07:53.587278 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:53.587401 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 16:07:53.588719 3724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:53.588273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:36811)
0929 16:07:53.588480 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:53.588577 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260929 16:07:53.588908 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 3558
W20260929 16:07:53.589035 3722 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 16:07:53.589774 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:38141
--local_ip_for_outbound_sockets=127.20.121.130
--tserver_master_addrs=127.20.121.190:38247
--webserver_port=38903
--webserver_interface=127.20.121.130
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:07:53.597263 3614 raft_consensus.cc:348] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:53.599133 3614 raft_consensus.cc:729] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18eb6e2c53b343fe8b3054da835182d0, State: Initialized, Role: FOLLOWER
I20260929 16:07:53.599864 3614 consensus_queue.cc:260] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:53.601835 3724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:53.601408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:40349)
0929 16:07:53.601590 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:53.601685 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 16:07:53.601830 3614 ts_tablet_manager.cc:1428] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0: Time spent starting tablet: real 0.017s user 0.014s sys 0.005s
I20260929 16:07:53.602977 3724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:53.602596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:49189)
0929 16:07:53.602756 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:53.602844 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 16:07:53.944989 3725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:53.945300 3725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:53.945398 3725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:53.953138 3725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:53.953284 3725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:53.953413 3725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:07:53.963711 3725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:38141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=38903
--tserver_master_addrs=127.20.121.190:38247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:53.965062 3725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:53.966429 3725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:53.981206 3734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:53.981161 3732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:53.981256 3731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:53.983896 3725 server_base.cc:1034] running on GCE node
I20260929 16:07:53.984674 3725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:53.985842 3725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:53.987100 3725 hybrid_clock.cc:648] HybridClock initialized: now 1790698073987045 us; error 75 us; skew 500 ppm
I20260929 16:07:53.987612 3725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:53 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698073, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:53.991648 3725 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:07:53.993566 3725 webserver.cc:460] Webserver started at http://127.20.121.130:38903/ using document root <none> and password file <none>
I20260929 16:07:53.994464 3725 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:53.994616 3725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:54.000960 3725 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 16:07:54.004415 3741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:54.006455 3725 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260929 16:07:54.006678 3725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "6eaf9b43fcec4d849eb22385dd22f1a8"
format_stamp: "Formatted at 2026-09-29 16:07:47 on dist-test-slave-x07d"
server_key: "779435239a1a9e3f56ca6c892fd15b69"
server_key_iv: "dc980135bc3a5f29be31df9655eaf1bc"
server_key_version: "encryptionkey@0"
I20260929 16:07:54.007938 3725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:54.031694 3725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:54.036964 3725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:54.037297 3725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:54.038388 3725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:54.039873 3725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:54.039961 3725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:54.040102 3725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:54.040203 3725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:54.075896 3725 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:38141
I20260929 16:07:54.076014 3853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:38141 every 8 connection(s)
I20260929 16:07:54.077663 3725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:07:54.079016 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 3725
I20260929 16:07:54.079701 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:43105
--local_ip_for_outbound_sockets=127.20.121.131
--tserver_master_addrs=127.20.121.190:38247
--webserver_port=38357
--webserver_interface=127.20.121.131
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:07:54.083174 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.080821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:33837)
0929 16:07:54.081665 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:54.081934 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":308,"threads_started":1}
W20260929 16:07:54.083920 3854 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:38247 (0 consecutive failures): Network error: Failed to ping master at 127.20.121.190:38247: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
I20260929 16:07:54.084920 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.084459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:39143)
0929 16:07:54.084660 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:54.084770 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260929 16:07:54.085805 3856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.085498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:52605)
0929 16:07:54.085628 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:54.085700 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 16:07:54.086139 3854 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 16:07:54.432571 3857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:07:54.432942 3857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:07:54.433153 3857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:07:54.441730 3857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:07:54.441927 3857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:07:54.442153 3857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:07:54.452625 3857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:43105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=38357
--tserver_master_addrs=127.20.121.190:38247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:07:54.453964 3857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:07:54.455415 3857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:07:54.469530 3862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:54.469511 3865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:07:54.469514 3863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:07:54.470106 3857 server_base.cc:1034] running on GCE node
I20260929 16:07:54.470922 3857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:07:54.471952 3857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:07:54.473210 3857 hybrid_clock.cc:648] HybridClock initialized: now 1790698074473140 us; error 66 us; skew 500 ppm
I20260929 16:07:54.473603 3857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:07:54 dist-test-slave-x07d krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698074, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:07:54.476975 3857 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:07:54.478519 3857 webserver.cc:460] Webserver started at http://127.20.121.131:38357/ using document root <none> and password file <none>
I20260929 16:07:54.479215 3857 fs_manager.cc:359] Metadata directory not provided
I20260929 16:07:54.479362 3857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:07:54.484961 3857 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 16:07:54.487970 3872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:54.489547 3857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:07:54.489732 3857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "62cbbbd02f9447dca873969bee33ca24"
format_stamp: "Formatted at 2026-09-29 16:07:48 on dist-test-slave-x07d"
server_key: "c6ce0e1167e46f3953e2934939af69d9"
server_key_iv: "7342659e2deb3325ba505eb583101dd4"
server_key_version: "encryptionkey@0"
I20260929 16:07:54.490886 3857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:07:54.506369 3857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:07:54.510833 3857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:07:54.511134 3857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:07:54.512159 3857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:07:54.513675 3857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:07:54.513800 3857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:54.513932 3857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:07:54.514042 3857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:07:54.549247 3857 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:43105
I20260929 16:07:54.549300 3984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:43105 every 8 connection(s)
I20260929 16:07:54.550818 3857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:07:54.555800 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 3857
I20260929 16:07:54.556052 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.553890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:56143)
0929 16:07:54.554668 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:54.554872 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":271,"threads_started":1}
W20260929 16:07:54.556917 3985 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:38247 (0 consecutive failures): Network error: Failed to ping master at 127.20.121.190:38247: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
I20260929 16:07:54.558123 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.557616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:48943)
0929 16:07:54.557862 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:54.557951 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260929 16:07:54.559077 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.558672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:51651)
0929 16:07:54.558890 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:54.558963 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 16:07:54.559350 3985 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 16:07:54.604734 3989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.603911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:55407)
0929 16:07:54.604408 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:54.604516 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":260,"threads_started":1}
I20260929 16:07:54.798278 3990 raft_consensus.cc:482] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:07:54.798604 3990 raft_consensus.cc:504] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:54.800300 3990 leader_election.cc:288] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141), 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105)
Sep 29 16:07:54 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698073, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
Sep 29 16:07:54 dist-test-slave-x07d krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698073, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:07:54.824409 3993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.800929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:57989 (local address 127.20.121.131:43105)
0929 16:07:54.802762 (+ 1833us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:54.802785 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:54.802842 (+ 57us) server_negotiation.cc:407] Connection header received
0929 16:07:54.804539 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:54.804549 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:54.804957 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:54.805237 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:54.807800 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:54.809130 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:54.811067 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:54.811837 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:54.812093 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:54.816619 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:54.816627 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:54.816637 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:54.816676 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:54.820286 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:54.821288 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:54.821294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:54.821299 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:54.821477 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:54.821958 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:54.821962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:54.821964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:54.822578 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:54.822794 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:54.823568 (+ 774us) server_negotiation.cc:299] Negotiation successful
0929 16:07:54.823978 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1589,"thread_start_us":1372,"threads_started":1}
I20260929 16:07:54.824419 3992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.800914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:36817 (local address 127.20.121.130:38141)
0929 16:07:54.802935 (+ 2021us) server_negotiation.cc:206] Beginning negotiation
0929 16:07:54.802951 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:07:54.803411 (+ 460us) server_negotiation.cc:407] Connection header received
0929 16:07:54.804976 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:07:54.804984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:07:54.805390 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:07:54.805711 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:07:54.807734 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:54.809039 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:54.811073 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:07:54.811756 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:07:54.812001 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:54.816377 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:07:54.816392 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:07:54.816419 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:07:54.816482 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:07:54.820202 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:54.821278 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:54.821286 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:54.821291 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:54.821468 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:07:54.821950 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:07:54.821957 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:07:54.821964 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:07:54.822586 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:07:54.822798 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0929 16:07:54.823607 (+ 809us) server_negotiation.cc:299] Negotiation successful
0929 16:07:54.823974 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1850,"thread_start_us":1736,"threads_started":1}
I20260929 16:07:54.824432 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.800714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:38141 (local address 127.20.121.129:36817)
0929 16:07:54.803274 (+ 2560us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:54.803290 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:54.803461 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:54.805723 (+ 2262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:54.805726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:54.806449 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:54.807500 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:54.807531 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:54.809244 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:54.809251 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:54.810905 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:54.810919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:54.811976 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:54.811983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:54.812259 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:54.813251 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:54.813269 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:54.815959 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:54.820408 (+ 4449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:54.820420 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:54.820439 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:54.821101 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:54.821585 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:54.821591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:54.821595 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:54.821822 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:54.822751 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:54.822756 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:54.823058 (+ 302us) client_negotiation.cc:769] Sending connection context
0929 16:07:54.823331 (+ 273us) client_negotiation.cc:240] Negotiation successful
0929 16:07:54.823961 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2458,"mutex_wait_us":293,"spinlock_wait_cycles":23168,"thread_start_us":2309,"threads_started":1}
I20260929 16:07:54.824867 3989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:54.800642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:43105 (local address 127.20.121.129:57989)
0929 16:07:54.800939 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:54.800989 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 16:07:54.801714 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:07:54.805467 (+ 3753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:07:54.805480 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:07:54.806452 (+ 972us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:07:54.807575 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:54.807598 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:54.809317 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:54.809321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:54.810908 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:07:54.810922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:07:54.812101 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:07:54.812107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:07:54.812277 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:07:54.813204 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:07:54.813228 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:07:54.816322 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:07:54.820479 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:54.820483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:54.820486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:54.821116 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:54.821622 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:07:54.821627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:07:54.821631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:07:54.821820 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:07:54.822720 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:07:54.822728 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:07:54.823057 (+ 329us) client_negotiation.cc:769] Sending connection context
0929 16:07:54.823295 (+ 238us) client_negotiation.cc:240] Negotiation successful
0929 16:07:54.824126 (+ 831us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":481,"spinlock_wait_cycles":24832}
W20260929 16:07:54.827772 3608 leader_election.cc:341] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105): Not found: Tablet not found: c4c06f9613654011994fdc656cac9d6f
W20260929 16:07:54.828092 3609 leader_election.cc:341] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141): Not found: Tablet not found: c4c06f9613654011994fdc656cac9d6f
I20260929 16:07:54.828262 3609 leader_election.cc:302] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18eb6e2c53b343fe8b3054da835182d0; no voters: 62cbbbd02f9447dca873969bee33ca24, 6eaf9b43fcec4d849eb22385dd22f1a8
I20260929 16:07:54.828707 3990 raft_consensus.cc:2726] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 16:07:55.087538 3994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:55.086643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:59397)
0929 16:07:55.087130 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:55.087272 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":237,"threads_started":1}
I20260929 16:07:55.560828 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:55.560013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:37847)
0929 16:07:55.560478 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:55.560602 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":221,"threads_started":1}
I20260929 16:07:55.606930 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:55.605955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:40423)
0929 16:07:55.606575 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:55.606671 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":301,"threads_started":1}
I20260929 16:07:56.089332 3997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:56.088596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:54697)
0929 16:07:56.089084 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:56.089173 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":260,"threads_started":1}
I20260929 16:07:56.562734 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:56.561823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:43179)
0929 16:07:56.562453 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:56.562545 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":461,"threads_started":1}
I20260929 16:07:56.608882 3999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:56.608013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:53895)
0929 16:07:56.608588 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:56.608681 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":228,"threads_started":1}
I20260929 16:07:56.949543 4000 raft_consensus.cc:482] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:07:56.949832 4000 raft_consensus.cc:504] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:56.950654 4000 leader_election.cc:288] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141), 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105)
W20260929 16:07:56.951777 3608 leader_election.cc:341] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105): Not found: Tablet not found: c4c06f9613654011994fdc656cac9d6f
W20260929 16:07:56.952001 3609 leader_election.cc:341] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141): Not found: Tablet not found: c4c06f9613654011994fdc656cac9d6f
I20260929 16:07:56.952143 3609 leader_election.cc:302] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18eb6e2c53b343fe8b3054da835182d0; no voters: 62cbbbd02f9447dca873969bee33ca24, 6eaf9b43fcec4d849eb22385dd22f1a8
I20260929 16:07:56.952443 4000 raft_consensus.cc:2726] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 16:07:57.091212 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:57.090411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:60745)
0929 16:07:57.090934 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:57.091033 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":241,"threads_started":1}
I20260929 16:07:57.564494 4002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:57.563727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:35299)
0929 16:07:57.564212 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:57.564320 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":268,"threads_started":1}
I20260929 16:07:57.610806 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:57.610046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:34937)
0929 16:07:57.610525 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:57.610615 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":252,"threads_started":1}
I20260929 16:07:58.093153 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:58.092312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:54955)
0929 16:07:58.092800 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:58.092962 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":241,"threads_started":1}
I20260929 16:07:58.566434 4005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:58.565587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:34871)
0929 16:07:58.566110 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:58.566205 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":248,"threads_started":1}
I20260929 16:07:58.612982 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:58.612168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:36469)
0929 16:07:58.612690 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:58.612775 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":316,"threads_started":1}
I20260929 16:07:59.095366 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:59.094330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:46889)
0929 16:07:59.094997 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:59.095115 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":340,"threads_started":1}
I20260929 16:07:59.226321 4008 raft_consensus.cc:482] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:07:59.226565 4008 raft_consensus.cc:504] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:07:59.227272 4008 leader_election.cc:288] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141), 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105)
W20260929 16:07:59.228312 3609 leader_election.cc:341] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141): Not found: Tablet not found: c4c06f9613654011994fdc656cac9d6f
W20260929 16:07:59.228547 3608 leader_election.cc:341] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105): Not found: Tablet not found: c4c06f9613654011994fdc656cac9d6f
I20260929 16:07:59.228732 3608 leader_election.cc:302] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18eb6e2c53b343fe8b3054da835182d0; no voters: 62cbbbd02f9447dca873969bee33ca24, 6eaf9b43fcec4d849eb22385dd22f1a8
I20260929 16:07:59.229056 4008 raft_consensus.cc:2726] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 16:07:59.568385 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:59.567536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:46275)
0929 16:07:59.568106 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:59.568208 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":294,"threads_started":1}
I20260929 16:07:59.615060 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:07:59.614163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:42429)
0929 16:07:59.614718 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0929 16:07:59.614832 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":257,"threads_started":1}
I20260929 16:08:00.097162 4011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:00.096432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:58599)
0929 16:08:00.096894 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:00.096983 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":234,"threads_started":1}
I20260929 16:08:00.570550 4012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:00.569442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:57367)
0929 16:08:00.569984 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:00.570346 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":278,"threads_started":1}
I20260929 16:08:00.616775 4013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:00.616008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:33851)
0929 16:08:00.616482 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:00.616578 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":248,"threads_started":1}
I20260929 16:08:00.822824 4014 raft_consensus.cc:482] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:08:00.823143 4014 raft_consensus.cc:504] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:08:00.823810 4014 leader_election.cc:288] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141), 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105)
W20260929 16:08:00.824924 3609 leader_election.cc:341] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141): Not found: Tablet not found: c4c06f9613654011994fdc656cac9d6f
W20260929 16:08:00.825088 3608 leader_election.cc:341] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105): Not found: Tablet not found: c4c06f9613654011994fdc656cac9d6f
I20260929 16:08:00.825235 3608 leader_election.cc:302] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18eb6e2c53b343fe8b3054da835182d0; no voters: 62cbbbd02f9447dca873969bee33ca24, 6eaf9b43fcec4d849eb22385dd22f1a8
I20260929 16:08:00.825529 4014 raft_consensus.cc:2726] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 16:08:01.099020 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:01.098223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:48315)
0929 16:08:01.098726 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:01.098803 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":237,"threads_started":1}
I20260929 16:08:01.572788 4016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:01.571884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:38615)
0929 16:08:01.572491 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:01.572582 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":276,"threads_started":1}
I20260929 16:08:01.618762 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:01.617838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:41711)
0929 16:08:01.618444 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:01.618560 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":317,"threads_started":1}
I20260929 16:08:02.100771 4018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:02.100028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:47091)
0929 16:08:02.100490 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:02.100578 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":240,"threads_started":1}
I20260929 16:08:02.574697 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:02.573870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:40651)
0929 16:08:02.574403 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:02.574505 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":303,"threads_started":1}
I20260929 16:08:02.620626 4020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:02.619822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:54705)
0929 16:08:02.620346 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:02.620441 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":299,"threads_started":1}
I20260929 16:08:03.102922 4021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:03.102068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:40725)
0929 16:08:03.102685 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:03.102764 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":269,"threads_started":1}
I20260929 16:08:03.576599 4022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:03.575763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:42689)
0929 16:08:03.576311 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:03.576434 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":272,"threads_started":1}
I20260929 16:08:03.622997 4023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:03.621981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:53987)
0929 16:08:03.622662 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:03.622764 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":331,"threads_started":1}
I20260929 16:08:04.104775 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:04.103849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:35003)
0929 16:08:04.104433 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:04.104542 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":312,"threads_started":1}
I20260929 16:08:04.578799 4025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:04.577762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:44023)
0929 16:08:04.578437 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:04.578543 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":408,"threads_started":1}
I20260929 16:08:04.625236 4026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:04.624244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:52555)
0929 16:08:04.624961 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:04.625049 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":316,"threads_started":1}
I20260929 16:08:05.106617 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:05.105813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:50115)
0929 16:08:05.106341 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:05.106456 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":292,"threads_started":1}
I20260929 16:08:05.580785 4028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:05.579940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:33557)
0929 16:08:05.580537 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:05.580623 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":313,"threads_started":1}
I20260929 16:08:05.627247 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:05.626369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:52643)
0929 16:08:05.626926 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:05.627033 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":253,"threads_started":1}
I20260929 16:08:06.108625 4030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:06.107684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:34631)
0929 16:08:06.108287 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:06.108384 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":308,"threads_started":1}
I20260929 16:08:06.582655 4031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:06.581768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:38481)
0929 16:08:06.582375 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:06.582478 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":310,"threads_started":1}
I20260929 16:08:06.629236 4032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:06.628405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:46777)
0929 16:08:06.628949 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:06.629052 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":257,"threads_started":1}
I20260929 16:08:07.110531 4033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:07.109668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:53353)
0929 16:08:07.110247 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:07.110351 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":254,"threads_started":1}
I20260929 16:08:07.584509 4034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:07.583733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:50285)
0929 16:08:07.584219 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:07.584340 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":255,"threads_started":1}
I20260929 16:08:07.631403 4035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:07.630595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:53755)
0929 16:08:07.631089 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:07.631187 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":272,"threads_started":1}
I20260929 16:08:07.754094 4036 raft_consensus.cc:482] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:08:07.754377 4036 raft_consensus.cc:504] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eaf9b43fcec4d849eb22385dd22f1a8" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 38141 } } peers { permanent_uuid: "18eb6e2c53b343fe8b3054da835182d0" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45329 } } peers { permanent_uuid: "62cbbbd02f9447dca873969bee33ca24" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 43105 } }
I20260929 16:08:07.755203 4036 leader_election.cc:288] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141), 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105)
W20260929 16:08:07.756523 3608 leader_election.cc:341] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62cbbbd02f9447dca873969bee33ca24 (127.20.121.131:43105): Not found: Tablet not found: c4c06f9613654011994fdc656cac9d6f
W20260929 16:08:07.756722 3609 leader_election.cc:341] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6eaf9b43fcec4d849eb22385dd22f1a8 (127.20.121.130:38141): Not found: Tablet not found: c4c06f9613654011994fdc656cac9d6f
I20260929 16:08:07.756898 3609 leader_election.cc:302] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 18eb6e2c53b343fe8b3054da835182d0; no voters: 62cbbbd02f9447dca873969bee33ca24, 6eaf9b43fcec4d849eb22385dd22f1a8
I20260929 16:08:07.757195 4036 raft_consensus.cc:2726] T c4c06f9613654011994fdc656cac9d6f P 18eb6e2c53b343fe8b3054da835182d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 16:08:08.112427 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:08.111542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:60657)
0929 16:08:08.112079 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:08.112170 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":290,"threads_started":1}
I20260929 16:08:08.586526 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:08.585665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:57671)
0929 16:08:08.586200 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:08.586316 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":264,"threads_started":1}
I20260929 16:08:08.633917 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:08.632836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.129:50429)
0929 16:08:08.633528 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:08.633649 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":297,"threads_started":1}
I20260929 16:08:09.114600 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:09.113630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.130:42323)
0929 16:08:09.114287 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:09.114437 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":401,"threads_started":1}
I20260929 16:08:09.557441 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 3558
I20260929 16:08:09.575217 4041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:09.558711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34526 (local address 127.20.121.129:45329)
0929 16:08:09.559554 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:09.559567 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:09.559607 (+ 40us) server_negotiation.cc:407] Connection header received
0929 16:08:09.559697 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:09.559707 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:09.560112 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:09.560301 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:09.562153 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:09.563210 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:09.564590 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:09.565350 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:09.565518 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:09.568531 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:09.568547 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:09.568562 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:09.568605 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:09.571924 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:09.572832 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:09.572840 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:09.572842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:09.572995 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:09.573434 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:09.573440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:09.573445 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:09.573865 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:09.574246 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:09.574580 (+ 334us) server_negotiation.cc:299] Negotiation successful
0929 16:08:09.574880 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":251,"threads_started":1}
I20260929 16:08:09.576642 3717 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:09.588768 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:09.587941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38247 (local address 127.20.121.131:44385)
0929 16:08:09.588490 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:09.588595 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":315,"threads_started":1}
I20260929 16:08:09.671566 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 3558
I20260929 16:08:09.690749 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 3725
I20260929 16:08:09.705314 4045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:09.691494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35120 (local address 127.20.121.130:38141)
0929 16:08:09.691996 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:09.692001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:09.692018 (+ 17us) server_negotiation.cc:407] Connection header received
0929 16:08:09.692085 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:09.692089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:09.692152 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:09.692246 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:09.693753 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:09.694668 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:09.696218 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:09.696671 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:09.696772 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:09.699887 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:09.699892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:09.699896 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:09.699938 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:09.702847 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:09.703541 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:09.703545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:09.703547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:09.703676 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:09.704069 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:09.704072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:09.704074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:09.704346 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:09.704569 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:09.704841 (+ 272us) server_negotiation.cc:299] Negotiation successful
0929 16:08:09.705027 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":290,"threads_started":1}
I20260929 16:08:09.706161 3849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:09.795169 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 3725
I20260929 16:08:09.813242 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 3857
I20260929 16:08:09.828277 4047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:09.814289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35150 (local address 127.20.121.131:43105)
0929 16:08:09.814931 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:09.814936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:09.814953 (+ 17us) server_negotiation.cc:407] Connection header received
0929 16:08:09.815009 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:09.815013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:09.815081 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:09.815170 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:09.816635 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:09.817458 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:09.818857 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:09.819388 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:09.819523 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:09.822318 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:09.822325 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:09.822329 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:09.822369 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:09.825514 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:09.826386 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:09.826392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:09.826402 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:09.826558 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:09.826880 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:09.826883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:09.826885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:09.827175 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:09.827414 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:09.827638 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 16:08:09.827826 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":289,"threads_started":1}
I20260929 16:08:09.829177 3980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:09.918042 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 3857
2026-09-29T16:08:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24968 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:08:10 dist-test-slave-x07d krb5kdc[4051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:08:10 dist-test-slave-x07d krb5kdc[4051](info): set up 2 sockets
Sep 29 16:08:10 dist-test-slave-x07d krb5kdc[4051](info): commencing operation
krb5kdc: starting...
W20260929 16:08:11.063145 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:08:11 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:08:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:08:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:08:11.317198 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34841
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:34841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:08:11.660837 4064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:11.661157 4064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:11.661334 4064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:11.668978 4064 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 16:08:11.669102 4064 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 16:08:11.669143 4064 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 16:08:11.669193 4064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:08:11.669293 4064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:11.669400 4064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:11.669485 4064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:08:11.669587 4064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:08:11.680830 4064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:34841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:11.682260 4064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:11.683635 4064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:11.696354 4072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:11.696323 4070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:11.697008 4069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:11.696973 4064 server_base.cc:1034] running on GCE node
I20260929 16:08:11.698165 4064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:11.699764 4064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:11.701058 4064 hybrid_clock.cc:648] HybridClock initialized: now 1790698091700984 us; error 36 us; skew 500 ppm
I20260929 16:08:11.701444 4064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:11 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698091, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:11.705178 4064 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:08:11.706848 4064 webserver.cc:460] Webserver started at http://127.20.121.190:34475/ using document root <none> and password file <none>
I20260929 16:08:11.707607 4064 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:11.707765 4064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:11.708099 4064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:08:11.711397 4064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "a592aebe041c41dab4a43d091e589bc8"
format_stamp: "Formatted at 2026-09-29 16:08:11 on dist-test-slave-x07d"
server_key: "e12a0a0e111b30a916ac8a7bad917f46"
server_key_iv: "8df6923090f8a165ee4c9aa9027e6d25"
server_key_version: "encryptionkey@0"
I20260929 16:08:11.712201 4064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "a592aebe041c41dab4a43d091e589bc8"
format_stamp: "Formatted at 2026-09-29 16:08:11 on dist-test-slave-x07d"
server_key: "e12a0a0e111b30a916ac8a7bad917f46"
server_key_iv: "8df6923090f8a165ee4c9aa9027e6d25"
server_key_version: "encryptionkey@0"
I20260929 16:08:11.717576 4064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 16:08:11.721426 4079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:11.723273 4064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 16:08:11.723490 4064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "a592aebe041c41dab4a43d091e589bc8"
format_stamp: "Formatted at 2026-09-29 16:08:11 on dist-test-slave-x07d"
server_key: "e12a0a0e111b30a916ac8a7bad917f46"
server_key_iv: "8df6923090f8a165ee4c9aa9027e6d25"
server_key_version: "encryptionkey@0"
I20260929 16:08:11.723754 4064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:11.757050 4064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:11.761900 4064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:11.762221 4064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:11.781003 4064 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:34841
I20260929 16:08:11.781039 4130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:34841 every 8 connection(s)
I20260929 16:08:11.782511 4064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:08:11.786669 4131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:08:11.792166 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 4064
I20260929 16:08:11.792395 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:08:11.793067 20966 external_mini_cluster.cc:1428] Setting key cb0020243b311a833c86a05187bb556c
I20260929 16:08:11.798985 4131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Bootstrap starting.
I20260929 16:08:11.802878 4131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Neither blocks nor log segments found. Creating new log.
I20260929 16:08:11.804088 4131 log.cc:824] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Log is configured to *not* fsync() on all Append() calls
I20260929 16:08:11.807696 4131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: No bootstrap required, opened a new log
Sep 29 16:08:11 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:08:11.816694 4131 raft_consensus.cc:348] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:11.817108 4131 raft_consensus.cc:374] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:08:11.817234 4131 raft_consensus.cc:729] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a592aebe041c41dab4a43d091e589bc8, State: Initialized, Role: FOLLOWER
I20260929 16:08:11.817957 4131 consensus_queue.cc:260] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:11.818300 4131 raft_consensus.cc:388] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:08:11.818476 4131 raft_consensus.cc:482] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:08:11.818678 4131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:08:11.819667 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:11.795359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40630 (local address 127.20.121.190:34841)
0929 16:08:11.796306 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:11.796323 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:11.796377 (+ 54us) server_negotiation.cc:407] Connection header received
0929 16:08:11.798846 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:11.798876 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:11.799399 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:11.799884 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:11.801412 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:11.803466 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:11.805574 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:11.806760 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:11.806978 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:11.811129 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:11.811141 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:11.811163 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:11.811224 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:11.815116 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:11.815900 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:11.815907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:11.815913 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:11.816087 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:11.816472 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:11.816480 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:11.816487 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:11.817089 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:11.817441 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:11.817866 (+ 425us) server_negotiation.cc:299] Negotiation successful
0929 16:08:11.818522 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":442,"threads_started":1}
I20260929 16:08:11.821707 4131 raft_consensus.cc:504] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:11.822314 4131 leader_election.cc:302] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a592aebe041c41dab4a43d091e589bc8; no voters:
I20260929 16:08:11.822996 4131 leader_election.cc:288] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:08:11.823176 4136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:08:11.823849 4136 raft_consensus.cc:686] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 1 LEADER]: Becoming Leader. State: Replica: a592aebe041c41dab4a43d091e589bc8, State: Running, Role: LEADER
I20260929 16:08:11.824359 4136 consensus_queue.cc:237] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:11.825043 4131 sys_catalog.cc:564] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:08:11.827224 4138 sys_catalog.cc:455] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a592aebe041c41dab4a43d091e589bc8. Latest consensus state: current_term: 1 leader_uuid: "a592aebe041c41dab4a43d091e589bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } } }
I20260929 16:08:11.827665 4138 sys_catalog.cc:458] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:08:11.828029 4137 sys_catalog.cc:455] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a592aebe041c41dab4a43d091e589bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } } }
I20260929 16:08:11.828343 4137 sys_catalog.cc:458] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:08:11.830951 4144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:08:11.835816 4144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:08:11.849468 4144 catalog_manager.cc:1347] Generated new cluster ID: 37a8b4aa9b0d4cfaa92076c0cacc9989
I20260929 16:08:11.849611 4144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:08:11.873632 4144 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:08:11.875672 4144 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:08:11.893596 4144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Generated new TSK 0
I20260929 16:08:11.894752 4144 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:08:11.962664 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34841
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:08:12.315186 4157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:12.315503 4157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:12.315665 4157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:12.323076 4157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:12.323232 4157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:12.323417 4157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:08:12.333935 4157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:12.335299 4157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:12.336608 4157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:12.350485 4165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:12.350564 4163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:12.350567 4162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:12.351599 4157 server_base.cc:1034] running on GCE node
I20260929 16:08:12.352298 4157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:12.353334 4157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:12.354708 4157 hybrid_clock.cc:648] HybridClock initialized: now 1790698092354633 us; error 63 us; skew 500 ppm
I20260929 16:08:12.355180 4157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:12 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698092, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:12.359246 4157 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:08:12.360997 4157 webserver.cc:460] Webserver started at http://127.20.121.129:46487/ using document root <none> and password file <none>
I20260929 16:08:12.361769 4157 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:12.361955 4157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:12.362308 4157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:08:12.365409 4157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "a8eca9e92a4d4b8b8f141fd623960677"
format_stamp: "Formatted at 2026-09-29 16:08:12 on dist-test-slave-x07d"
server_key: "6a0f05cc983c9f9fcf6251a26cc80cc3"
server_key_iv: "ce34475799aee60c9b84f1e7bc4b98af"
server_key_version: "encryptionkey@0"
I20260929 16:08:12.366246 4157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "a8eca9e92a4d4b8b8f141fd623960677"
format_stamp: "Formatted at 2026-09-29 16:08:12 on dist-test-slave-x07d"
server_key: "6a0f05cc983c9f9fcf6251a26cc80cc3"
server_key_iv: "ce34475799aee60c9b84f1e7bc4b98af"
server_key_version: "encryptionkey@0"
I20260929 16:08:12.372584 4157 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260929 16:08:12.376453 4172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:12.378327 4157 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:08:12.378530 4157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "a8eca9e92a4d4b8b8f141fd623960677"
format_stamp: "Formatted at 2026-09-29 16:08:12 on dist-test-slave-x07d"
server_key: "6a0f05cc983c9f9fcf6251a26cc80cc3"
server_key_iv: "ce34475799aee60c9b84f1e7bc4b98af"
server_key_version: "encryptionkey@0"
I20260929 16:08:12.378733 4157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:12.401852 4157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:12.406313 4157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:12.406567 4157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:12.407497 4157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:08:12.408818 4157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:08:12.408895 4157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:12.408987 4157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:08:12.409030 4157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:12.445169 4157 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:43085
I20260929 16:08:12.445358 4284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:43085 every 8 connection(s)
I20260929 16:08:12.446619 4157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:08:12.454130 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 4157
I20260929 16:08:12.454401 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:08:12.455245 20966 external_mini_cluster.cc:1428] Setting key 40252fe6b216b5b5e5487b8846e226e9
Sep 29 16:08:12 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698092, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:08:12.479684 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:12.449462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:41977 (local address 127.20.121.190:34841)
0929 16:08:12.449919 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:12.449924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:12.451171 (+ 1247us) server_negotiation.cc:407] Connection header received
0929 16:08:12.453188 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:12.453195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:12.453350 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:12.453533 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:12.455952 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:12.456749 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:12.458712 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:12.459546 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:12.459700 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:12.472599 (+ 12899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:12.472606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:12.472610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:12.472655 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:12.475822 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:12.476814 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:12.476819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:12.476822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:12.476956 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:12.477499 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:12.477504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:12.477506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:12.477821 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:12.478062 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:12.478978 (+ 916us) server_negotiation.cc:299] Negotiation successful
0929 16:08:12.479235 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":219,"threads_started":1}
I20260929 16:08:12.480814 4288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:12.450196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:41977)
0929 16:08:12.450992 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:12.451036 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:12.452864 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:12.453799 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:12.453812 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:12.454732 (+ 920us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:08:12.455686 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:12.455734 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:12.456952 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:12.456957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:12.458547 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:12.458560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:12.459834 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:12.459845 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:12.460080 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:12.469027 (+ 8947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:08:12.469072 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:08:12.472232 (+ 3160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:08:12.476040 (+ 3808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:12.476052 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:12.476075 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:12.476621 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:12.477092 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:12.477098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:12.477103 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:12.477342 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:12.477980 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:08:12.477990 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:08:12.478458 (+ 468us) client_negotiation.cc:769] Sending connection context
0929 16:08:12.478792 (+ 334us) client_negotiation.cc:240] Negotiation successful
0929 16:08:12.479435 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":300,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:08:12.483021 4285 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34841
I20260929 16:08:12.483384 4285 heartbeater.cc:464] Registering TS with master...
I20260929 16:08:12.484338 4285 heartbeater.cc:511] Master 127.20.121.190:34841 requested a full tablet report, sending...
I20260929 16:08:12.487214 4096 ts_manager.cc:194] Registered new tserver with Master: a8eca9e92a4d4b8b8f141fd623960677 (127.20.121.129:43085)
I20260929 16:08:12.489508 4096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:41977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:08:12.519472 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34841
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:08:12.874740 4291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:12.875109 4291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:12.875299 4291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:12.883074 4291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:12.883247 4291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:12.883437 4291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:08:12.894593 4291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:12.895979 4291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:12.897336 4291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:12.912617 4296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:12.912663 4297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:12.912621 4299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:12.913673 4291 server_base.cc:1034] running on GCE node
I20260929 16:08:12.914512 4291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:12.915647 4291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:12.916985 4291 hybrid_clock.cc:648] HybridClock initialized: now 1790698092916886 us; error 93 us; skew 500 ppm
I20260929 16:08:12.917363 4291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:12 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698092, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:12.921074 4291 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:08:12.922938 4291 webserver.cc:460] Webserver started at http://127.20.121.130:39691/ using document root <none> and password file <none>
I20260929 16:08:12.923729 4291 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:12.923892 4291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:12.924226 4291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:08:12.927533 4291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "6969c39eddcc48edacc462a4a6e8234e"
format_stamp: "Formatted at 2026-09-29 16:08:12 on dist-test-slave-x07d"
server_key: "c421d90491b1660039cb6c43432f8907"
server_key_iv: "49cea2febfc0ae6c6d2ecddab4d002a1"
server_key_version: "encryptionkey@0"
I20260929 16:08:12.928443 4291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "6969c39eddcc48edacc462a4a6e8234e"
format_stamp: "Formatted at 2026-09-29 16:08:12 on dist-test-slave-x07d"
server_key: "c421d90491b1660039cb6c43432f8907"
server_key_iv: "49cea2febfc0ae6c6d2ecddab4d002a1"
server_key_version: "encryptionkey@0"
I20260929 16:08:12.934312 4291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 16:08:12.938354 4306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:12.940016 4291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:08:12.940256 4291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "6969c39eddcc48edacc462a4a6e8234e"
format_stamp: "Formatted at 2026-09-29 16:08:12 on dist-test-slave-x07d"
server_key: "c421d90491b1660039cb6c43432f8907"
server_key_iv: "49cea2febfc0ae6c6d2ecddab4d002a1"
server_key_version: "encryptionkey@0"
I20260929 16:08:12.940491 4291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:12.960778 4291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:12.965669 4291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:12.966050 4291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:12.967073 4291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:08:12.968484 4291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:08:12.968592 4291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:12.968756 4291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:08:12.968832 4291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:13.004004 4291 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:44925
I20260929 16:08:13.004055 4418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:44925 every 8 connection(s)
I20260929 16:08:13.005510 4291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:08:13.015957 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 4291
I20260929 16:08:13.016197 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:08:13.016932 20966 external_mini_cluster.cc:1428] Setting key ee0bf32ebb9b4c2a13e146696905a32d
Sep 29 16:08:13 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698092, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:08:13.029141 4421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.008247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:40991 (local address 127.20.121.190:34841)
0929 16:08:13.008757 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:13.008765 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:13.009705 (+ 940us) server_negotiation.cc:407] Connection header received
0929 16:08:13.011627 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:13.011632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:13.011702 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:13.011812 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:13.013438 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.014406 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.016440 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.017148 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.017351 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.021723 (+ 4372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:13.021732 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:13.021741 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:13.021790 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:13.025574 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:13.026598 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:13.026607 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:13.026613 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:13.026786 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:13.027256 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:13.027265 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:13.027271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:13.027624 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:13.027797 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:13.028523 (+ 726us) server_negotiation.cc:299] Negotiation successful
0929 16:08:13.028748 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":247,"threads_started":1}
I20260929 16:08:13.030375 4422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.008813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:40991)
0929 16:08:13.009531 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:13.009574 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:13.011333 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:13.012070 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:13.012078 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:13.012608 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:08:13.013231 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.013250 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.014614 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.014619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.016224 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.016241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.017449 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.017456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.017703 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.018699 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:08:13.018810 (+ 111us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:08:13.021470 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:08:13.025733 (+ 4263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:13.025740 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:13.025755 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:13.026462 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:13.026902 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:13.026905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:13.026907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:13.027119 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:13.027779 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:08:13.027788 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:08:13.028159 (+ 371us) client_negotiation.cc:769] Sending connection context
0929 16:08:13.028440 (+ 281us) client_negotiation.cc:240] Negotiation successful
0929 16:08:13.029016 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":271,"threads_started":1}
I20260929 16:08:13.031986 4419 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34841
I20260929 16:08:13.032332 4419 heartbeater.cc:464] Registering TS with master...
I20260929 16:08:13.033249 4419 heartbeater.cc:511] Master 127.20.121.190:34841 requested a full tablet report, sending...
I20260929 16:08:13.035090 4096 ts_manager.cc:194] Registered new tserver with Master: 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925)
I20260929 16:08:13.036011 4096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:40991
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:08:13.077418 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34841
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:08:13.427430 4425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:13.427832 4425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:13.428004 4425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:13.436338 4425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:13.436486 4425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:13.436614 4425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:08:13.447422 4425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:13.448837 4425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:13.450273 4425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:13.464406 4433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:13.464475 4431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:13.464519 4430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:13.465446 4425 server_base.cc:1034] running on GCE node
I20260929 16:08:13.466300 4425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:13.467361 4425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:13.468640 4425 hybrid_clock.cc:648] HybridClock initialized: now 1790698093468597 us; error 61 us; skew 500 ppm
I20260929 16:08:13.469020 4425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:13 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698093, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:13.472285 4425 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:08:13.473887 4425 webserver.cc:460] Webserver started at http://127.20.121.131:46385/ using document root <none> and password file <none>
I20260929 16:08:13.474677 4425 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:13.474833 4425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:13.475149 4425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:08:13.478266 4425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "2a7c348c04374ec39a644c8cf268ead6"
format_stamp: "Formatted at 2026-09-29 16:08:13 on dist-test-slave-x07d"
server_key: "b91de395e758016fc1a0b7d9e18e55c3"
server_key_iv: "f1bb16cf4e90cef23afffa4844c8e323"
server_key_version: "encryptionkey@0"
I20260929 16:08:13.479072 4425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "2a7c348c04374ec39a644c8cf268ead6"
format_stamp: "Formatted at 2026-09-29 16:08:13 on dist-test-slave-x07d"
server_key: "b91de395e758016fc1a0b7d9e18e55c3"
server_key_iv: "f1bb16cf4e90cef23afffa4844c8e323"
server_key_version: "encryptionkey@0"
I20260929 16:08:13.484829 4425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:08:13.488744 4440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:13.490839 4425 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:08:13.491055 4425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "2a7c348c04374ec39a644c8cf268ead6"
format_stamp: "Formatted at 2026-09-29 16:08:13 on dist-test-slave-x07d"
server_key: "b91de395e758016fc1a0b7d9e18e55c3"
server_key_iv: "f1bb16cf4e90cef23afffa4844c8e323"
server_key_version: "encryptionkey@0"
I20260929 16:08:13.491291 4425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:13.493228 4285 heartbeater.cc:503] Master 127.20.121.190:34841 was elected leader, sending a full tablet report...
I20260929 16:08:13.518841 4425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:13.523512 4425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:13.523845 4425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:13.524988 4425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:08:13.526510 4425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:08:13.526615 4425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:13.526772 4425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:08:13.526844 4425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:13.563568 4425 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:40311
I20260929 16:08:13.563659 4552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:40311 every 8 connection(s)
I20260929 16:08:13.564906 4425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:08:13.572734 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 4425
I20260929 16:08:13.572958 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:08:13.573513 20966 external_mini_cluster.cc:1428] Setting key 9337c9bfcd722b45eb8a9df3cba47fe9
Sep 29 16:08:13 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698093, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:08:13.587975 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.567747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:48801 (local address 127.20.121.190:34841)
0929 16:08:13.568291 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:13.568298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:13.569367 (+ 1069us) server_negotiation.cc:407] Connection header received
0929 16:08:13.571434 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:13.571438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:13.571526 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:13.571630 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:13.573842 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.574853 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.577038 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.577450 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.577569 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.581344 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:13.581350 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:13.581354 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:13.581391 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:13.584517 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:13.585427 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:13.585431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:13.585433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:13.585566 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:13.586106 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:13.586109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:13.586112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:13.586398 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:13.586529 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:13.587323 (+ 794us) server_negotiation.cc:299] Negotiation successful
0929 16:08:13.587563 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":275,"threads_started":1}
I20260929 16:08:13.589104 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.568402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:48801)
0929 16:08:13.569188 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:13.569229 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:13.571048 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:13.571944 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:13.571953 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:13.572735 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:08:13.573619 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.573649 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.575038 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.575044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.576869 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.576887 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.577680 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.577685 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.577918 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.578728 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:08:13.578752 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:08:13.581092 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:08:13.584711 (+ 3619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:13.584720 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:13.584736 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:13.585305 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:13.585663 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:13.585666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:13.585669 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:13.585932 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:13.586530 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:08:13.586537 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:08:13.586946 (+ 409us) client_negotiation.cc:769] Sending connection context
0929 16:08:13.587252 (+ 306us) client_negotiation.cc:240] Negotiation successful
0929 16:08:13.587831 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":325,"threads_started":1}
I20260929 16:08:13.590694 4553 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34841
I20260929 16:08:13.591046 4553 heartbeater.cc:464] Registering TS with master...
I20260929 16:08:13.591809 4553 heartbeater.cc:511] Master 127.20.121.190:34841 requested a full tablet report, sending...
I20260929 16:08:13.593428 4096 ts_manager.cc:194] Registered new tserver with Master: 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311)
I20260929 16:08:13.594373 4096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:48801
I20260929 16:08:13.602901 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:08:13.622713 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.607755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.20.121.190:34841)
0929 16:08:13.607935 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:13.607940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:13.608341 (+ 401us) server_negotiation.cc:407] Connection header received
0929 16:08:13.608495 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:13.608499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:13.608571 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:13.608678 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:13.610362 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.611165 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.612826 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.613276 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.613388 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.616593 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:13.616598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:13.616602 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:13.616653 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:13.619898 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:13.620733 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:13.620737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:13.620739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:13.620874 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:13.621263 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:13.621267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:13.621269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:13.621620 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:13.621829 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:13.622283 (+ 454us) server_negotiation.cc:299] Negotiation successful
0929 16:08:13.622484 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 16:08:13.627766 4096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:08:13.636232 4096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:08:13.667822 4568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.650361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:44925 (local address 127.0.0.1:40822)
0929 16:08:13.655587 (+ 5226us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:13.655629 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:13.655824 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:13.657051 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:13.657059 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:13.657088 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:08:13.657536 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.657554 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.659385 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.659403 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.664568 (+ 5165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.664593 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.666732 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.666740 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.666978 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.667039 (+ 61us) client_negotiation.cc:769] Sending connection context
0929 16:08:13.667183 (+ 144us) client_negotiation.cc:240] Negotiation successful
0929 16:08:13.667392 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4946,"mutex_wait_us":72,"thread_start_us":5776,"threads_started":1}
I20260929 16:08:13.668188 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.651504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40612 (local address 127.20.121.131:40311)
0929 16:08:13.652243 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:13.652257 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:13.656528 (+ 4271us) server_negotiation.cc:407] Connection header received
0929 16:08:13.656827 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:13.656837 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:13.657101 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:13.657336 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:08:13.659621 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.661797 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.665306 (+ 3509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.666965 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.667196 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.667366 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:13.667619 (+ 253us) server_negotiation.cc:299] Negotiation successful
0929 16:08:13.667792 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":407,"threads_started":1}
I20260929 16:08:13.668905 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.650551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40822 (local address 127.20.121.130:44925)
0929 16:08:13.656376 (+ 5825us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:13.656387 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:13.656419 (+ 32us) server_negotiation.cc:407] Connection header received
0929 16:08:13.656526 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:13.656533 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:13.656818 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:13.657037 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:08:13.657719 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.659237 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.664827 (+ 5590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.666515 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.666701 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.666904 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:13.668360 (+ 1456us) server_negotiation.cc:299] Negotiation successful
0929 16:08:13.668586 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5652,"thread_start_us":5514,"threads_started":1}
I20260929 16:08:13.669296 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.650886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40311 (local address 127.0.0.1:40612)
0929 16:08:13.656407 (+ 5521us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:13.656423 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:13.656631 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:13.658777 (+ 2146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:13.658786 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:13.658827 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:08:13.659295 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.659311 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.662131 (+ 2820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.662140 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.665108 (+ 2968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.665124 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.667197 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.667204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.667405 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.667438 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 16:08:13.667559 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 16:08:13.668885 (+ 1326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5381,"mutex_wait_us":1,"thread_start_us":4266,"threads_started":1}
I20260929 16:08:13.672837 4488 tablet_service.cc:1467] Processing CreateTablet for tablet cdb53203befd47ac8064e7ec35846edd (DEFAULT_TABLE table=test-table [id=246145a627e44ce2837aa3e6d23473e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:08:13.672699 4354 tablet_service.cc:1467] Processing CreateTablet for tablet cdb53203befd47ac8064e7ec35846edd (DEFAULT_TABLE table=test-table [id=246145a627e44ce2837aa3e6d23473e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:08:13.674407 4488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdb53203befd47ac8064e7ec35846edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:08:13.674602 4354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdb53203befd47ac8064e7ec35846edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:08:13.675202 4571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.657283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.0.0.1:52792)
0929 16:08:13.665187 (+ 7904us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:13.665206 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:13.665334 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:13.665933 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:13.665938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:13.665965 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:08:13.666406 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.666416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.668315 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.668320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.671809 (+ 3489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.671836 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.673912 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.673920 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.674387 (+ 467us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.674443 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 16:08:13.674616 (+ 173us) client_negotiation.cc:240] Negotiation successful
0929 16:08:13.674767 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7762,"mutex_wait_us":484,"thread_start_us":7505,"threads_started":1}
I20260929 16:08:13.675999 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.656781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52792 (local address 127.20.121.129:43085)
0929 16:08:13.664860 (+ 8079us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:13.664869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:13.665312 (+ 443us) server_negotiation.cc:407] Connection header received
0929 16:08:13.665495 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:13.665501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:13.665720 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:13.665943 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:08:13.666570 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.668137 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.672071 (+ 3934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.673647 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.673884 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.674130 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:13.675471 (+ 1341us) server_negotiation.cc:299] Negotiation successful
0929 16:08:13.675663 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7870,"thread_start_us":7682,"threads_started":1}
I20260929 16:08:13.686244 4220 tablet_service.cc:1467] Processing CreateTablet for tablet cdb53203befd47ac8064e7ec35846edd (DEFAULT_TABLE table=test-table [id=246145a627e44ce2837aa3e6d23473e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:08:13.686945 4572 tablet_bootstrap.cc:492] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: Bootstrap starting.
I20260929 16:08:13.688236 4220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cdb53203befd47ac8064e7ec35846edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:08:13.691502 4572 tablet_bootstrap.cc:654] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: Neither blocks nor log segments found. Creating new log.
I20260929 16:08:13.692806 4572 log.cc:824] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: Log is configured to *not* fsync() on all Append() calls
I20260929 16:08:13.695288 4573 tablet_bootstrap.cc:492] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: Bootstrap starting.
I20260929 16:08:13.696672 4572 tablet_bootstrap.cc:492] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: No bootstrap required, opened a new log
I20260929 16:08:13.697003 4572 ts_tablet_manager.cc:1397] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260929 16:08:13.699286 4573 tablet_bootstrap.cc:654] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: Neither blocks nor log segments found. Creating new log.
I20260929 16:08:13.700642 4573 log.cc:824] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: Log is configured to *not* fsync() on all Append() calls
I20260929 16:08:13.703986 4576 tablet_bootstrap.cc:492] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: Bootstrap starting.
I20260929 16:08:13.704274 4573 tablet_bootstrap.cc:492] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: No bootstrap required, opened a new log
I20260929 16:08:13.704579 4573 ts_tablet_manager.cc:1397] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260929 16:08:13.708158 4572 raft_consensus.cc:348] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } } peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } }
I20260929 16:08:13.708525 4576 tablet_bootstrap.cc:654] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: Neither blocks nor log segments found. Creating new log.
I20260929 16:08:13.708669 4572 raft_consensus.cc:374] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:08:13.708808 4572 raft_consensus.cc:729] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6969c39eddcc48edacc462a4a6e8234e, State: Initialized, Role: FOLLOWER
I20260929 16:08:13.709515 4572 consensus_queue.cc:260] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } } peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } }
I20260929 16:08:13.709942 4576 log.cc:824] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: Log is configured to *not* fsync() on all Append() calls
I20260929 16:08:13.710748 4419 heartbeater.cc:503] Master 127.20.121.190:34841 was elected leader, sending a full tablet report...
I20260929 16:08:13.711702 4572 ts_tablet_manager.cc:1428] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260929 16:08:13.713987 4573 raft_consensus.cc:348] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } } peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } }
I20260929 16:08:13.714511 4573 raft_consensus.cc:374] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:08:13.714694 4573 raft_consensus.cc:729] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a7c348c04374ec39a644c8cf268ead6, State: Initialized, Role: FOLLOWER
I20260929 16:08:13.715546 4573 consensus_queue.cc:260] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } } peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } }
I20260929 16:08:13.717399 4573 ts_tablet_manager.cc:1428] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: Time spent starting tablet: real 0.013s user 0.006s sys 0.005s
I20260929 16:08:13.717693 4576 tablet_bootstrap.cc:492] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: No bootstrap required, opened a new log
I20260929 16:08:13.717864 4553 heartbeater.cc:503] Master 127.20.121.190:34841 was elected leader, sending a full tablet report...
I20260929 16:08:13.718080 4576 ts_tablet_manager.cc:1397] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260929 16:08:13.726089 4576 raft_consensus.cc:348] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } } peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } }
I20260929 16:08:13.726495 4576 raft_consensus.cc:374] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:08:13.726600 4576 raft_consensus.cc:729] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8eca9e92a4d4b8b8f141fd623960677, State: Initialized, Role: FOLLOWER
I20260929 16:08:13.727257 4576 consensus_queue.cc:260] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } } peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } }
I20260929 16:08:13.729055 4576 ts_tablet_manager.cc:1428] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260929 16:08:13.754616 4580 raft_consensus.cc:482] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:08:13.754882 4580 raft_consensus.cc:504] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } } peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } }
I20260929 16:08:13.756599 4580 leader_election.cc:288] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925), 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311)
W20260929 16:08:13.757800 4420 tablet.cc:2369] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:08:13.764056 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.757207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:35405 (local address 127.20.121.131:40311)
0929 16:08:13.757385 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:13.757393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:13.758256 (+ 863us) server_negotiation.cc:407] Connection header received
0929 16:08:13.758592 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:13.758596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:13.758660 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:13.758764 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:08:13.759379 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.760161 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.762235 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.763029 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.763226 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.763310 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:13.763657 (+ 347us) server_negotiation.cc:299] Negotiation successful
0929 16:08:13.763797 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 16:08:13.764215 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.757129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:47685 (local address 127.20.121.130:44925)
0929 16:08:13.757355 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:13.757362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:13.758251 (+ 889us) server_negotiation.cc:407] Connection header received
0929 16:08:13.758572 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:13.758577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:13.758667 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:13.758790 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:08:13.759539 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.760271 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.762319 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.763174 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.763369 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.763454 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:13.763809 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 16:08:13.763942 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 16:08:13.764308 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.756891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40311 (local address 127.20.121.129:35405)
0929 16:08:13.758118 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:13.758142 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:13.758416 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:13.758779 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:13.758783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:13.758817 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:08:13.759214 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.759226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.760302 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.760308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.762061 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.762082 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.763240 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.763246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.763454 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.763518 (+ 64us) client_negotiation.cc:769] Sending connection context
0929 16:08:13.763632 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 16:08:13.763773 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":94,"spinlock_wait_cycles":40704,"thread_start_us":1155,"threads_started":1}
I20260929 16:08:13.764372 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.757110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:44925 (local address 127.20.121.129:47685)
0929 16:08:13.758127 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:13.758149 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:13.758382 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:13.758905 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:13.758912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:13.758959 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:08:13.759349 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.759357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.760424 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.760431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.762155 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:13.762169 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.763400 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.763406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:13.763650 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.763674 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 16:08:13.763781 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 16:08:13.763914 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":66,"thread_start_us":506,"threads_started":1}
I20260929 16:08:13.765435 4374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdb53203befd47ac8064e7ec35846edd" candidate_uuid: "a8eca9e92a4d4b8b8f141fd623960677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6969c39eddcc48edacc462a4a6e8234e" is_pre_election: true
I20260929 16:08:13.765442 4508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdb53203befd47ac8064e7ec35846edd" candidate_uuid: "a8eca9e92a4d4b8b8f141fd623960677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7c348c04374ec39a644c8cf268ead6" is_pre_election: true
I20260929 16:08:13.765954 4508 raft_consensus.cc:2454] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8eca9e92a4d4b8b8f141fd623960677 in term 0.
I20260929 16:08:13.765952 4374 raft_consensus.cc:2454] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8eca9e92a4d4b8b8f141fd623960677 in term 0.
I20260929 16:08:13.766798 4175 leader_election.cc:302] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6969c39eddcc48edacc462a4a6e8234e, a8eca9e92a4d4b8b8f141fd623960677; no voters:
I20260929 16:08:13.767158 4580 raft_consensus.cc:2781] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:08:13.767333 4580 raft_consensus.cc:482] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:08:13.767475 4580 raft_consensus.cc:3037] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:08:13.769670 4580 raft_consensus.cc:504] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } } peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } }
I20260929 16:08:13.770363 4580 leader_election.cc:288] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 1 election: Requested vote from peers 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925), 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311)
I20260929 16:08:13.770869 4374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdb53203befd47ac8064e7ec35846edd" candidate_uuid: "a8eca9e92a4d4b8b8f141fd623960677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6969c39eddcc48edacc462a4a6e8234e"
I20260929 16:08:13.770946 4508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cdb53203befd47ac8064e7ec35846edd" candidate_uuid: "a8eca9e92a4d4b8b8f141fd623960677" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7c348c04374ec39a644c8cf268ead6"
I20260929 16:08:13.771081 4374 raft_consensus.cc:3037] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:08:13.771188 4508 raft_consensus.cc:3037] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:08:13.773348 4374 raft_consensus.cc:2454] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8eca9e92a4d4b8b8f141fd623960677 in term 1.
I20260929 16:08:13.773422 4508 raft_consensus.cc:2454] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8eca9e92a4d4b8b8f141fd623960677 in term 1.
I20260929 16:08:13.773819 4175 leader_election.cc:302] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6969c39eddcc48edacc462a4a6e8234e, a8eca9e92a4d4b8b8f141fd623960677; no voters:
I20260929 16:08:13.774179 4580 raft_consensus.cc:2781] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:08:13.774660 4580 raft_consensus.cc:686] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 1 LEADER]: Becoming Leader. State: Replica: a8eca9e92a4d4b8b8f141fd623960677, State: Running, Role: LEADER
I20260929 16:08:13.775221 4580 consensus_queue.cc:237] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } } peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } }
I20260929 16:08:13.780231 4096 catalog_manager.cc:5495] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 reported cstate change: term changed from 0 to 1, leader changed from <none> to a8eca9e92a4d4b8b8f141fd623960677 (127.20.121.129). New cstate: current_term: 1 leader_uuid: "a8eca9e92a4d4b8b8f141fd623960677" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } health_report { overall_health: UNKNOWN } } }
W20260929 16:08:13.816615 4554 tablet.cc:2369] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:08:13.818547 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.813098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.20.121.129:43085)
0929 16:08:13.813313 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:13.813319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:13.813338 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:08:13.813528 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:13.813533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:13.813610 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:13.813767 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:08:13.814219 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.814940 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.816316 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.816758 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.816923 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.817433 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:08:13.817937 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:08:13.818141 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:13.818193 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 16:08:13.818278 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 16:08:13.826880 4374 raft_consensus.cc:1261] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e [term 1 FOLLOWER]: Refusing update from remote peer a8eca9e92a4d4b8b8f141fd623960677: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:08:13.826880 4508 raft_consensus.cc:1261] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6 [term 1 FOLLOWER]: Refusing update from remote peer a8eca9e92a4d4b8b8f141fd623960677: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:08:13.827952 4583 consensus_queue.cc:1035] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:08:13.828337 4580 consensus_queue.cc:1035] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:08:13.840534 4586 mvcc.cc:205] Tried to move back new op lower bound from 7334699392304087040 to 7334699392107749376. Current Snapshot: MvccSnapshot[applied={T|T < 7334699392304087040}]
I20260929 16:08:13.841522 4587 mvcc.cc:205] Tried to move back new op lower bound from 7334699392304087040 to 7334699392107749376. Current Snapshot: MvccSnapshot[applied={T|T < 7334699392304087040}]
I20260929 16:08:13.854750 4096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:08:13.855036 4096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:08:13.859242 4096 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Sending DeleteTablet for 3 replicas of tablet cdb53203befd47ac8064e7ec35846edd
I20260929 16:08:13.860687 4488 tablet_service.cc:1514] Processing DeleteTablet for tablet cdb53203befd47ac8064e7ec35846edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:08:13 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:40612
I20260929 16:08:13.860700 4354 tablet_service.cc:1514] Processing DeleteTablet for tablet cdb53203befd47ac8064e7ec35846edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:08:13 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:40822
I20260929 16:08:13.860843 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 4157
I20260929 16:08:13.861802 4220 tablet_service.cc:1514] Processing DeleteTablet for tablet cdb53203befd47ac8064e7ec35846edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:08:13 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:52792
I20260929 16:08:13.874065 4596 tablet_replica.cc:330] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: stopping tablet replica
I20260929 16:08:13.874251 4595 tablet_replica.cc:330] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: stopping tablet replica
I20260929 16:08:13.874769 4595 raft_consensus.cc:2229] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:08:13.874740 4596 raft_consensus.cc:2229] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:08:13.875362 4595 raft_consensus.cc:2258] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:08:13.875387 4596 raft_consensus.cc:2258] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:08:13.877774 4598 tablet_replica.cc:330] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: stopping tablet replica
I20260929 16:08:13.877794 4595 ts_tablet_manager.cc:1905] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:08:13.878506 4598 raft_consensus.cc:2229] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:08:13.880038 4598 raft_consensus.cc:2258] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:08:13.880661 4596 ts_tablet_manager.cc:1905] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:08:13.885236 4595 ts_tablet_manager.cc:1918] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:08:13.885475 4595 log.cc:1196] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/cdb53203befd47ac8064e7ec35846edd
I20260929 16:08:13.885780 4598 ts_tablet_manager.cc:1905] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:08:13.886107 4595 ts_tablet_manager.cc:1939] T cdb53203befd47ac8064e7ec35846edd P 2a7c348c04374ec39a644c8cf268ead6: Deleting consensus metadata
I20260929 16:08:13.888180 4596 ts_tablet_manager.cc:1918] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:08:13.888134 4081 catalog_manager.cc:4841] TS 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311): tablet cdb53203befd47ac8064e7ec35846edd (table test-table [id=246145a627e44ce2837aa3e6d23473e5]) successfully deleted
I20260929 16:08:13.888394 4596 log.cc:1196] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/cdb53203befd47ac8064e7ec35846edd
I20260929 16:08:13.888960 4596 ts_tablet_manager.cc:1939] T cdb53203befd47ac8064e7ec35846edd P 6969c39eddcc48edacc462a4a6e8234e: Deleting consensus metadata
I20260929 16:08:13.890892 4082 catalog_manager.cc:4841] TS 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925): tablet cdb53203befd47ac8064e7ec35846edd (table test-table [id=246145a627e44ce2837aa3e6d23473e5]) successfully deleted
Sep 29 16:08:13 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:08:13.896462 4598 ts_tablet_manager.cc:1918] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:08:13.896662 4598 log.cc:1196] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/cdb53203befd47ac8064e7ec35846edd
I20260929 16:08:13.897296 4598 ts_tablet_manager.cc:1939] T cdb53203befd47ac8064e7ec35846edd P a8eca9e92a4d4b8b8f141fd623960677: Deleting consensus metadata
I20260929 16:08:13.899516 4082 catalog_manager.cc:4841] TS a8eca9e92a4d4b8b8f141fd623960677 (127.20.121.129:43085): tablet cdb53203befd47ac8064e7ec35846edd (table test-table [id=246145a627e44ce2837aa3e6d23473e5]) successfully deleted
I20260929 16:08:13.903996 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:13.861714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52806 (local address 127.20.121.129:43085)
0929 16:08:13.861948 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:13.861957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:13.879334 (+ 17377us) server_negotiation.cc:407] Connection header received
0929 16:08:13.880567 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:13.880573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:13.880658 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:13.880750 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:13.886107 (+ 5357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.887309 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.890129 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:13.890881 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:13.891141 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:13.897060 (+ 5919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:13.897078 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:13.897101 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:13.897152 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:13.900752 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:13.901735 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:13.901744 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:13.901746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:13.901926 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:13.902442 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:13.902446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:13.902448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:13.902851 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:13.903078 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:13.903474 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 16:08:13.903666 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 16:08:13.904814 4280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:14.005971 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 4157
I20260929 16:08:14.025543 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 4291
Sep 29 16:08:14 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:08:14.042214 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:14.026450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40838 (local address 127.20.121.130:44925)
0929 16:08:14.026648 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:14.026660 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:14.026678 (+ 18us) server_negotiation.cc:407] Connection header received
0929 16:08:14.026755 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:14.026759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:14.026827 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:14.026911 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:14.028461 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:14.029377 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:14.030908 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:14.031308 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:14.031411 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:14.035931 (+ 4520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:14.035948 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:14.035971 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:14.036023 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:14.039129 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:14.039913 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:14.039921 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:14.039923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:14.040058 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:14.040484 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:14.040488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:14.040490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:14.040869 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:14.041193 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:14.041553 (+ 360us) server_negotiation.cc:299] Negotiation successful
0929 16:08:14.041778 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 16:08:14.042997 4414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:14.135057 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 4291
I20260929 16:08:14.154933 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 4425
Sep 29 16:08:14 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:08:14.173060 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:14.156716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.20.121.131:40311)
0929 16:08:14.157161 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:14.157169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:14.157194 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:08:14.157269 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:14.157273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:14.157422 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:14.157558 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:14.159018 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:14.159797 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:14.161215 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:14.161629 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:14.161726 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:14.166864 (+ 5138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:14.166881 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:14.166898 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:14.166937 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:14.169950 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:14.170999 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:14.171006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:14.171008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:14.171142 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:14.171541 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:14.171548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:14.171550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:14.171995 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:14.172270 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:14.172600 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 16:08:14.172785 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260929 16:08:14.173776 4548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:14.265785 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 4425
I20260929 16:08:14.282879 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 4064
I20260929 16:08:14.284123 4126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:14.398635 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 4064
I20260929 16:08:34.421172 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34841
--webserver_interface=127.20.121.190
--webserver_port=34475
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:34841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:08:34.769800 4603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:34.770231 4603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:34.770385 4603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:34.777972 4603 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 16:08:34.778182 4603 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 16:08:34.778252 4603 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 16:08:34.778331 4603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:08:34.778414 4603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:34.778537 4603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:34.778607 4603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:08:34.778692 4603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:08:34.789227 4603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:34841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=34475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:34.790776 4603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:34.792214 4603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:34.817746 4612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:34.817839 4609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:34.817776 4610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:34.818835 4603 server_base.cc:1034] running on GCE node
I20260929 16:08:34.819867 4603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:34.822126 4603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:34.823426 4603 hybrid_clock.cc:648] HybridClock initialized: now 1790698114823355 us; error 38 us; skew 500 ppm
I20260929 16:08:34.823980 4603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:34 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698114, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:34.832167 4603 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:08:34.834398 4603 webserver.cc:460] Webserver started at http://127.20.121.190:34475/ using document root <none> and password file <none>
I20260929 16:08:34.835356 4603 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:34.835536 4603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:34.842662 4603 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 16:08:34.846329 4619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:34.847985 4603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:08:34.848306 4603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "a592aebe041c41dab4a43d091e589bc8"
format_stamp: "Formatted at 2026-09-29 16:08:11 on dist-test-slave-x07d"
server_key: "e12a0a0e111b30a916ac8a7bad917f46"
server_key_iv: "8df6923090f8a165ee4c9aa9027e6d25"
server_key_version: "encryptionkey@0"
I20260929 16:08:34.849445 4603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:34.874521 4603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:34.879254 4603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:34.879599 4603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:34.899586 4603 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:34841
I20260929 16:08:34.899609 4670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:34841 every 8 connection(s)
I20260929 16:08:34.901250 4603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:08:34.908628 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 4603
I20260929 16:08:34.909624 4671 sys_catalog.cc:263] Verifying existing consensus state
I20260929 16:08:34.909744 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:43085
--local_ip_for_outbound_sockets=127.20.121.129
--tserver_master_addrs=127.20.121.190:34841
--webserver_port=46487
--webserver_interface=127.20.121.129
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:08:34.915598 4671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Bootstrap starting.
I20260929 16:08:34.937444 4671 log.cc:824] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Log is configured to *not* fsync() on all Append() calls
I20260929 16:08:34.951615 4671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 16:08:34.952232 4671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Bootstrap complete.
I20260929 16:08:34.962416 4671 raft_consensus.cc:348] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:34.963340 4671 raft_consensus.cc:729] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a592aebe041c41dab4a43d091e589bc8, State: Initialized, Role: FOLLOWER
I20260929 16:08:34.964059 4671 consensus_queue.cc:260] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:34.964336 4671 raft_consensus.cc:388] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:08:34.964478 4671 raft_consensus.cc:482] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:08:34.964666 4671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 1 FOLLOWER]: Advancing to term 2
I20260929 16:08:34.968734 4671 raft_consensus.cc:504] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:34.969312 4671 leader_election.cc:302] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a592aebe041c41dab4a43d091e589bc8; no voters:
I20260929 16:08:34.970180 4671 leader_election.cc:288] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 16:08:34.970327 4675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 16:08:34.971582 4675 raft_consensus.cc:686] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 2 LEADER]: Becoming Leader. State: Replica: a592aebe041c41dab4a43d091e589bc8, State: Running, Role: LEADER
I20260929 16:08:34.972173 4675 consensus_queue.cc:237] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:34.973019 4671 sys_catalog.cc:564] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:08:34.975186 4677 sys_catalog.cc:455] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a592aebe041c41dab4a43d091e589bc8. Latest consensus state: current_term: 2 leader_uuid: "a592aebe041c41dab4a43d091e589bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } } }
I20260929 16:08:34.976009 4677 sys_catalog.cc:458] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:08:34.975754 4676 sys_catalog.cc:455] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a592aebe041c41dab4a43d091e589bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } } }
I20260929 16:08:34.976501 4676 sys_catalog.cc:458] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:08:34.987836 4682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:08:34.997211 4682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:08:34.998878 4682 catalog_manager.cc:1259] Loaded cluster ID: 37a8b4aa9b0d4cfaa92076c0cacc9989
I20260929 16:08:34.999030 4682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:08:35.003628 4682 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:08:35.005674 4682 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Loaded TSK: 0
I20260929 16:08:35.059707 4682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Generated new TSK 1
I20260929 16:08:35.060242 4682 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 16:08:35.299494 4673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:35.299865 4673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:35.300034 4673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:35.307871 4673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:35.308039 4673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:35.308218 4673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:08:35.318557 4673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:43085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=46487
--tserver_master_addrs=127.20.121.190:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:35.320075 4673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:35.321434 4673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:35.338155 4701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:35.338157 4699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:35.338886 4698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:35.339143 4673 server_base.cc:1034] running on GCE node
I20260929 16:08:35.339895 4673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:35.340996 4673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:35.342338 4673 hybrid_clock.cc:648] HybridClock initialized: now 1790698115342263 us; error 56 us; skew 500 ppm
I20260929 16:08:35.342759 4673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:35 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698115, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:35.347487 4673 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:08:35.349490 4673 webserver.cc:460] Webserver started at http://127.20.121.129:46487/ using document root <none> and password file <none>
I20260929 16:08:35.350545 4673 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:35.350724 4673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:35.359020 4673 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 16:08:35.362993 4708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:35.364805 4673 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260929 16:08:35.365001 4673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "a8eca9e92a4d4b8b8f141fd623960677"
format_stamp: "Formatted at 2026-09-29 16:08:12 on dist-test-slave-x07d"
server_key: "6a0f05cc983c9f9fcf6251a26cc80cc3"
server_key_iv: "ce34475799aee60c9b84f1e7bc4b98af"
server_key_version: "encryptionkey@0"
I20260929 16:08:35.366460 4673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:35.383867 4673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:35.390134 4673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:35.390462 4673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:35.391846 4673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:08:35.393747 4673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:08:35.393862 4673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:35.393958 4673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:08:35.394061 4673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:35.432318 4673 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:43085
I20260929 16:08:35.432391 4820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:43085 every 8 connection(s)
I20260929 16:08:35.434454 4673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:08:35.439772 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 4673
I20260929 16:08:35.440584 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:44925
--local_ip_for_outbound_sockets=127.20.121.130
--tserver_master_addrs=127.20.121.190:34841
--webserver_port=39691
--webserver_interface=127.20.121.130
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:08:35 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698115, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:08:35.468721 4824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:35.438711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:49119)
0929 16:08:35.439868 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:35.439918 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:35.442227 (+ 2309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:35.445038 (+ 2811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:35.445047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:35.445678 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:08:35.446643 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:35.446668 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:35.449806 (+ 3138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:35.449817 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:35.452436 (+ 2619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:35.452469 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:35.454730 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:35.454742 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:35.455096 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:35.456007 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:08:35.456047 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:08:35.459105 (+ 3058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:08:35.463739 (+ 4634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:35.463750 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:35.463776 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:35.464294 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:35.464848 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:35.464853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:35.464857 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:35.465080 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:35.465952 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:08:35.465961 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:08:35.466387 (+ 426us) client_negotiation.cc:769] Sending connection context
0929 16:08:35.466692 (+ 305us) client_negotiation.cc:240] Negotiation successful
0929 16:08:35.467313 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"thread_start_us":463,"threads_started":1}
I20260929 16:08:35.468721 4823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:35.438416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:49119 (local address 127.20.121.190:34841)
0929 16:08:35.439279 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:35.439299 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:35.440088 (+ 789us) server_negotiation.cc:407] Connection header received
0929 16:08:35.443197 (+ 3109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:35.443220 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:35.444189 (+ 969us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:35.444753 (+ 564us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:35.446997 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:35.449489 (+ 2492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:35.452853 (+ 3364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:35.454316 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:35.454519 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:35.459506 (+ 4987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:35.459516 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:35.459532 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:35.459587 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:35.463507 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:35.464498 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:35.464514 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:35.464519 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:35.464706 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:35.465223 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:35.465228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:35.465232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:35.465782 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:35.466056 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:35.466960 (+ 904us) server_negotiation.cc:299] Negotiation successful
0929 16:08:35.467384 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":381,"threads_started":1}
I20260929 16:08:35.472610 4821 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34841
I20260929 16:08:35.473129 4821 heartbeater.cc:464] Registering TS with master...
I20260929 16:08:35.474159 4821 heartbeater.cc:511] Master 127.20.121.190:34841 requested a full tablet report, sending...
I20260929 16:08:35.476984 4636 ts_manager.cc:194] Registered new tserver with Master: a8eca9e92a4d4b8b8f141fd623960677 (127.20.121.129:43085)
I20260929 16:08:35.478626 4636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:49119
W20260929 16:08:35.800770 4825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:35.801142 4825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:35.801304 4825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:35.809204 4825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:35.809376 4825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:35.809549 4825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:08:35.820096 4825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:44925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=39691
--tserver_master_addrs=127.20.121.190:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:35.821462 4825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:35.822868 4825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:35.837778 4830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:35.837821 4831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:35.837978 4833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:35.838681 4825 server_base.cc:1034] running on GCE node
I20260929 16:08:35.839468 4825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:35.840569 4825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:35.841847 4825 hybrid_clock.cc:648] HybridClock initialized: now 1790698115841777 us; error 64 us; skew 500 ppm
I20260929 16:08:35.842216 4825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:35 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698115, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:35.845634 4825 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:08:35.847442 4825 webserver.cc:460] Webserver started at http://127.20.121.130:39691/ using document root <none> and password file <none>
I20260929 16:08:35.848305 4825 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:35.848529 4825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:35.855203 4825 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 16:08:35.858467 4840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:35.860150 4825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 16:08:35.860392 4825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "6969c39eddcc48edacc462a4a6e8234e"
format_stamp: "Formatted at 2026-09-29 16:08:12 on dist-test-slave-x07d"
server_key: "c421d90491b1660039cb6c43432f8907"
server_key_iv: "49cea2febfc0ae6c6d2ecddab4d002a1"
server_key_version: "encryptionkey@0"
I20260929 16:08:35.861580 4825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:35.880406 4825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:35.885628 4825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:35.885985 4825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:35.887192 4825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:08:35.888825 4825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:08:35.888971 4825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:35.889128 4825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:08:35.889230 4825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:35.927838 4825 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:44925
I20260929 16:08:35.927968 4952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:44925 every 8 connection(s)
I20260929 16:08:35.929510 4825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:08:35.932293 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 4825
I20260929 16:08:35.933122 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:40311
--local_ip_for_outbound_sockets=127.20.121.131
--tserver_master_addrs=127.20.121.190:34841
--webserver_port=46385
--webserver_interface=127.20.121.131
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 16:08:35 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698115, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:08:35.956280 4823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:35.932938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:42991 (local address 127.20.121.190:34841)
0929 16:08:35.933136 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:35.933140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:35.934839 (+ 1699us) server_negotiation.cc:407] Connection header received
0929 16:08:35.937237 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:35.937243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:35.937396 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:35.937520 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:35.939419 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:35.940519 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:35.942470 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:35.943272 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:35.943445 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:35.947870 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:35.947879 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:35.947887 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:35.947941 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:35.951914 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:35.952954 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:35.952962 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:35.952969 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:35.953148 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:35.953836 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:35.953844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:35.953850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:35.954344 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:35.954589 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:35.955464 (+ 875us) server_negotiation.cc:299] Negotiation successful
0929 16:08:35.955785 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 16:08:35.956959 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:35.933620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:42991)
0929 16:08:35.934592 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:35.934644 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:35.936794 (+ 2150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:35.937800 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:35.937810 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:35.938517 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:08:35.939172 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:35.939191 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:35.940760 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:35.940765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:35.942264 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:35.942281 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:35.943558 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:35.943564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:35.943809 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:35.944683 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:08:35.944709 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:08:35.947502 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:08:35.952176 (+ 4674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:35.952186 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:35.952203 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:35.952756 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:35.953437 (+ 681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:35.953442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:35.953445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:35.953673 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:35.954532 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:08:35.954540 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:08:35.954982 (+ 442us) client_negotiation.cc:769] Sending connection context
0929 16:08:35.955293 (+ 311us) client_negotiation.cc:240] Negotiation successful
0929 16:08:35.955744 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":443,"threads_started":1}
I20260929 16:08:35.958855 4953 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34841
I20260929 16:08:35.959190 4953 heartbeater.cc:464] Registering TS with master...
I20260929 16:08:35.959975 4953 heartbeater.cc:511] Master 127.20.121.190:34841 requested a full tablet report, sending...
I20260929 16:08:35.961592 4636 ts_manager.cc:194] Registered new tserver with Master: 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925)
I20260929 16:08:35.962654 4636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:42991
W20260929 16:08:36.281425 4956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:36.281788 4956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:36.281965 4956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:36.289435 4956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:36.289616 4956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:36.289812 4956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:08:36.299837 4956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:40311
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=46385
--tserver_master_addrs=127.20.121.190:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:36.301368 4956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:36.302816 4956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:36.316767 4964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:36.316887 4961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:36.316918 4962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:36.317615 4956 server_base.cc:1034] running on GCE node
I20260929 16:08:36.318399 4956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:36.319671 4956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:36.320941 4956 hybrid_clock.cc:648] HybridClock initialized: now 1790698116320866 us; error 60 us; skew 500 ppm
I20260929 16:08:36.321349 4956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:36 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698116, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:36.324738 4956 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:08:36.326323 4956 webserver.cc:460] Webserver started at http://127.20.121.131:46385/ using document root <none> and password file <none>
I20260929 16:08:36.327065 4956 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:36.327226 4956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:36.332844 4956 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260929 16:08:36.335734 4971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:36.337200 4956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:08:36.337416 4956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "2a7c348c04374ec39a644c8cf268ead6"
format_stamp: "Formatted at 2026-09-29 16:08:13 on dist-test-slave-x07d"
server_key: "b91de395e758016fc1a0b7d9e18e55c3"
server_key_iv: "f1bb16cf4e90cef23afffa4844c8e323"
server_key_version: "encryptionkey@0"
I20260929 16:08:36.338524 4956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:36.369841 4956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:36.374732 4956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:36.375023 4956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:36.376029 4956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:08:36.377434 4956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:08:36.377553 4956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:36.377691 4956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:08:36.377782 4956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:36.411994 4956 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:40311
I20260929 16:08:36.412086 5083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:40311 every 8 connection(s)
I20260929 16:08:36.413547 4956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:08:36.421886 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 4956
Sep 29 16:08:36 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698116, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:08:36.436354 4823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.416334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:37007 (local address 127.20.121.190:34841)
0929 16:08:36.416561 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.416568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.417717 (+ 1149us) server_negotiation.cc:407] Connection header received
0929 16:08:36.419737 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.419742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.419811 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.419912 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:36.421517 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.422619 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.424610 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.425048 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.425187 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.429293 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:36.429297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:36.429301 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:36.429348 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:36.433148 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:36.433930 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:36.433939 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:36.433944 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:36.434132 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:36.434579 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:36.434585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:36.434591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:36.434883 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:36.435031 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:36.435716 (+ 685us) server_negotiation.cc:299] Negotiation successful
0929 16:08:36.435950 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260929 16:08:36.437597 5086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.416838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:37007)
0929 16:08:36.417544 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:36.417589 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:36.419481 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:36.420173 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:36.420182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:36.420705 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:08:36.421289 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.421306 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.422786 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.422798 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.424443 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.424468 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.425279 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.425287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.425538 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.426308 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:08:36.426338 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:08:36.429062 (+ 2724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:08:36.433296 (+ 4234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:36.433307 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:36.433324 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:36.433787 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:36.434244 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:36.434249 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:36.434253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:36.434462 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:36.435020 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:08:36.435028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:08:36.435361 (+ 333us) client_negotiation.cc:769] Sending connection context
0929 16:08:36.435637 (+ 276us) client_negotiation.cc:240] Negotiation successful
0929 16:08:36.436208 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":275,"threads_started":1}
I20260929 16:08:36.439147 5084 heartbeater.cc:347] Connected to a master server at 127.20.121.190:34841
I20260929 16:08:36.439510 5084 heartbeater.cc:464] Registering TS with master...
I20260929 16:08:36.440382 5084 heartbeater.cc:511] Master 127.20.121.190:34841 requested a full tablet report, sending...
I20260929 16:08:36.442241 4636 ts_manager.cc:194] Registered new tserver with Master: 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311)
I20260929 16:08:36.443161 5088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.422934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50494 (local address 127.20.121.190:34841)
0929 16:08:36.426261 (+ 3327us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.426267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.426287 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:08:36.426339 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.426344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.426437 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.426555 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:36.428514 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.429530 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.431587 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.432319 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.432456 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.436121 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:36.436128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:36.436132 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:36.436168 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:36.440040 (+ 3872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:36.440843 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:36.440848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:36.440851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:36.441003 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:36.441442 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:36.441445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:36.441447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:36.441862 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:36.442176 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:36.442535 (+ 359us) server_negotiation.cc:299] Negotiation successful
0929 16:08:36.442765 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3169,"thread_start_us":2989,"threads_started":1}
I20260929 16:08:36.443349 4636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:37007
I20260929 16:08:36.446364 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:08:36.453809 5088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.447339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.20.121.190:34841)
0929 16:08:36.447535 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.447540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.448016 (+ 476us) server_negotiation.cc:407] Connection header received
0929 16:08:36.448249 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.448254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.448400 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.448527 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:08:36.448974 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.449697 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.451637 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.452193 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.452309 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.452999 (+ 690us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:08:36.453263 (+ 264us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 16:08:36.453587 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50498: token expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260929 16:08:36.453999 5088 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50498: token expired
W20260929 16:08:36.454818 4558 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.121.190:34841)
I20260929 16:08:36.469059 5088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.455371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50504 (local address 127.20.121.190:34841)
0929 16:08:36.455554 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.455558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.455574 (+ 16us) server_negotiation.cc:407] Connection header received
0929 16:08:36.455672 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.455676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.455738 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.455823 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:36.457405 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.458270 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.459717 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.460200 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.460314 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.463325 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:36.463330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:36.463333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:36.463372 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:36.466354 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:36.467157 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:36.467162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:36.467164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:36.467308 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:36.467686 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:36.467689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:36.467691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:36.468078 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:36.468281 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:36.468583 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 16:08:36.468770 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 16:08:36.473052 4636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:08:36.481062 4636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:08:36.482236 4821 heartbeater.cc:503] Master 127.20.121.190:34841 was elected leader, sending a full tablet report...
I20260929 16:08:36.506107 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.495694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33538 (local address 127.20.121.130:44925)
0929 16:08:36.498232 (+ 2538us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.498240 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.498270 (+ 30us) server_negotiation.cc:407] Connection header received
0929 16:08:36.498366 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.498376 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.498636 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.498865 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:08:36.499586 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.501174 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.503462 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.504733 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.504946 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.505153 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:36.505550 (+ 397us) server_negotiation.cc:299] Negotiation successful
0929 16:08:36.505735 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2380,"thread_start_us":2249,"threads_started":1}
I20260929 16:08:36.506247 5097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.495126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:44925 (local address 127.0.0.1:33538)
0929 16:08:36.497473 (+ 2347us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:36.497498 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:36.497660 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:36.498871 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:36.498879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:36.498926 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:08:36.499340 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.499351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.501342 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.501347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.503274 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.503287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.504995 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.505001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.505243 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.505308 (+ 65us) client_negotiation.cc:769] Sending connection context
0929 16:08:36.505501 (+ 193us) client_negotiation.cc:240] Negotiation successful
0929 16:08:36.505759 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2152,"mutex_wait_us":32,"thread_start_us":1980,"threads_started":1}
I20260929 16:08:36.506367 5095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.494974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36374 (local address 127.20.121.131:40311)
0929 16:08:36.496366 (+ 1392us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.496375 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.496681 (+ 306us) server_negotiation.cc:407] Connection header received
0929 16:08:36.497028 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.497040 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.497335 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.498273 (+ 938us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:08:36.498569 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.500308 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.503450 (+ 3142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.505045 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.505607 (+ 562us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.505783 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:36.505854 (+ 71us) server_negotiation.cc:299] Negotiation successful
0929 16:08:36.506038 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1198,"thread_start_us":980,"threads_started":1}
I20260929 16:08:36.506760 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.494604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40311 (local address 127.0.0.1:36374)
0929 16:08:36.496521 (+ 1917us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:36.496558 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:36.496785 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:36.497570 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:36.497577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:36.497619 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:08:36.498125 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.498141 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.500529 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.500537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.503254 (+ 2717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.503270 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.505330 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.505337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.505624 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.505655 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 16:08:36.505739 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 16:08:36.505879 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1689,"mutex_wait_us":51,"thread_start_us":1529,"threads_started":1}
I20260929 16:08:36.509737 5099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.498082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36622 (local address 127.20.121.129:43085)
0929 16:08:36.499554 (+ 1472us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.499563 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.500331 (+ 768us) server_negotiation.cc:407] Connection header received
0929 16:08:36.500451 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.500461 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.500698 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.500902 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:08:36.501601 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.503227 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.507007 (+ 3780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.508367 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.508616 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.508793 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:36.509251 (+ 458us) server_negotiation.cc:299] Negotiation successful
0929 16:08:36.509422 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1279,"thread_start_us":1131,"threads_started":1}
I20260929 16:08:36.509778 5098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.497551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.0.0.1:36622)
0929 16:08:36.500071 (+ 2520us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:36.500089 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:36.500203 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:36.500889 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:36.500894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:36.500920 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:08:36.501394 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.501405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.503408 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.503413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.506796 (+ 3383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.506813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.508925 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.508930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.509055 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.509079 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 16:08:36.509194 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 16:08:36.509351 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2383,"mutex_wait_us":25,"thread_start_us":2228,"threads_started":1}
I20260929 16:08:36.511102 4888 tablet_service.cc:1467] Processing CreateTablet for tablet f64a4a389e6b410b9709e85820e1af70 (DEFAULT_TABLE table=test-table [id=1de8cbfb233a46cf891c6bd00bb3bb78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:08:36.511814 5019 tablet_service.cc:1467] Processing CreateTablet for tablet f64a4a389e6b410b9709e85820e1af70 (DEFAULT_TABLE table=test-table [id=1de8cbfb233a46cf891c6bd00bb3bb78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:08:36.512786 4888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64a4a389e6b410b9709e85820e1af70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:08:36.513252 5019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64a4a389e6b410b9709e85820e1af70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:08:36.514212 4756 tablet_service.cc:1467] Processing CreateTablet for tablet f64a4a389e6b410b9709e85820e1af70 (DEFAULT_TABLE table=test-table [id=1de8cbfb233a46cf891c6bd00bb3bb78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:08:36.515786 4756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64a4a389e6b410b9709e85820e1af70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:08:36.528319 5100 tablet_bootstrap.cc:492] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: Bootstrap starting.
I20260929 16:08:36.529448 5101 tablet_bootstrap.cc:492] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: Bootstrap starting.
I20260929 16:08:36.533342 5101 tablet_bootstrap.cc:654] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: Neither blocks nor log segments found. Creating new log.
I20260929 16:08:36.533473 5102 tablet_bootstrap.cc:492] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Bootstrap starting.
I20260929 16:08:36.533725 5100 tablet_bootstrap.cc:654] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: Neither blocks nor log segments found. Creating new log.
I20260929 16:08:36.534945 5100 log.cc:824] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: Log is configured to *not* fsync() on all Append() calls
I20260929 16:08:36.534948 5101 log.cc:824] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: Log is configured to *not* fsync() on all Append() calls
I20260929 16:08:36.538159 5101 tablet_bootstrap.cc:492] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: No bootstrap required, opened a new log
I20260929 16:08:36.538264 5102 tablet_bootstrap.cc:654] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Neither blocks nor log segments found. Creating new log.
I20260929 16:08:36.538468 5101 ts_tablet_manager.cc:1397] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260929 16:08:36.538533 5100 tablet_bootstrap.cc:492] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: No bootstrap required, opened a new log
I20260929 16:08:36.538801 5100 ts_tablet_manager.cc:1397] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260929 16:08:36.539597 5102 log.cc:824] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Log is configured to *not* fsync() on all Append() calls
I20260929 16:08:36.544801 5102 tablet_bootstrap.cc:492] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: No bootstrap required, opened a new log
I20260929 16:08:36.545112 5102 ts_tablet_manager.cc:1397] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.004s
I20260929 16:08:36.547423 5101 raft_consensus.cc:348] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:36.547833 5101 raft_consensus.cc:374] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:08:36.547940 5101 raft_consensus.cc:729] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6969c39eddcc48edacc462a4a6e8234e, State: Initialized, Role: FOLLOWER
I20260929 16:08:36.548578 5101 consensus_queue.cc:260] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:36.549733 4953 heartbeater.cc:503] Master 127.20.121.190:34841 was elected leader, sending a full tablet report...
I20260929 16:08:36.549551 5100 raft_consensus.cc:348] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:36.549976 5100 raft_consensus.cc:374] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:08:36.550088 5100 raft_consensus.cc:729] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a7c348c04374ec39a644c8cf268ead6, State: Initialized, Role: FOLLOWER
I20260929 16:08:36.550496 5101 ts_tablet_manager.cc:1428] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260929 16:08:36.550657 5100 consensus_queue.cc:260] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:36.552389 5084 heartbeater.cc:503] Master 127.20.121.190:34841 was elected leader, sending a full tablet report...
I20260929 16:08:36.553061 5100 ts_tablet_manager.cc:1428] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260929 16:08:36.558234 5102 raft_consensus.cc:348] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:36.558657 5102 raft_consensus.cc:374] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:08:36.558769 5102 raft_consensus.cc:729] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8eca9e92a4d4b8b8f141fd623960677, State: Initialized, Role: FOLLOWER
I20260929 16:08:36.559466 5102 consensus_queue.cc:260] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:36.561327 5102 ts_tablet_manager.cc:1428] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260929 16:08:36.665369 5085 tablet.cc:2369] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:08:36.682271 4954 tablet.cc:2369] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:08:36.687502 4822 tablet.cc:2369] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:08:36.788743 5107 raft_consensus.cc:482] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:08:36.789077 5107 raft_consensus.cc:504] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:36.791064 5107 leader_election.cc:288] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925), a8eca9e92a4d4b8b8f141fd623960677 (127.20.121.129:43085)
I20260929 16:08:36.798513 5086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.791543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:44925 (local address 127.20.121.131:57311)
0929 16:08:36.791846 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:36.791869 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:36.792028 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:36.792636 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:36.792641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:36.792678 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:08:36.792890 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.792904 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.794311 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.794315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.796183 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.796198 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.797718 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.797723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.797918 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.797952 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 16:08:36.798073 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 16:08:36.798220 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":1}
I20260929 16:08:36.798625 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.791584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:57311 (local address 127.20.121.130:44925)
0929 16:08:36.791856 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.791935 (+ 79us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.791960 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:08:36.792256 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.792263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.792363 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.792515 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:08:36.793053 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.794166 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.796369 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.797502 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.797683 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.797768 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:36.798121 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 16:08:36.798309 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 16:08:36.799614 5109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.792023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.20.121.131:58931)
0929 16:08:36.792883 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:36.792900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:36.793025 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:36.793372 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:36.793376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:36.793399 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:08:36.793675 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.793682 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.795002 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.795007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.796818 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:36.796828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.798852 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.798859 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:36.799051 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.799098 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 16:08:36.799246 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 16:08:36.799352 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":468,"threads_started":1}
I20260929 16:08:36.799732 5099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.792293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:58931 (local address 127.20.121.129:43085)
0929 16:08:36.792523 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.792529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.793021 (+ 492us) server_negotiation.cc:407] Connection header received
0929 16:08:36.793131 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.793137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.793219 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.793326 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:08:36.793919 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.794824 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.796999 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.798430 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.798599 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.798690 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:36.799311 (+ 621us) server_negotiation.cc:299] Negotiation successful
0929 16:08:36.799486 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 16:08:36.799650 4908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a4a389e6b410b9709e85820e1af70" candidate_uuid: "2a7c348c04374ec39a644c8cf268ead6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6969c39eddcc48edacc462a4a6e8234e" is_pre_election: true
I20260929 16:08:36.800280 4908 raft_consensus.cc:2454] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a7c348c04374ec39a644c8cf268ead6 in term 0.
I20260929 16:08:36.800762 4776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a4a389e6b410b9709e85820e1af70" candidate_uuid: "2a7c348c04374ec39a644c8cf268ead6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8eca9e92a4d4b8b8f141fd623960677" is_pre_election: true
I20260929 16:08:36.800979 4974 leader_election.cc:302] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a7c348c04374ec39a644c8cf268ead6, 6969c39eddcc48edacc462a4a6e8234e; no voters:
I20260929 16:08:36.801422 5107 raft_consensus.cc:2781] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:08:36.801373 4776 raft_consensus.cc:2454] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a7c348c04374ec39a644c8cf268ead6 in term 0.
I20260929 16:08:36.801596 5107 raft_consensus.cc:482] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:08:36.801714 5107 raft_consensus.cc:3037] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:08:36.804284 5107 raft_consensus.cc:504] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:36.804996 5107 leader_election.cc:288] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [CANDIDATE]: Term 1 election: Requested vote from peers 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925), a8eca9e92a4d4b8b8f141fd623960677 (127.20.121.129:43085)
I20260929 16:08:36.805531 4908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a4a389e6b410b9709e85820e1af70" candidate_uuid: "2a7c348c04374ec39a644c8cf268ead6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6969c39eddcc48edacc462a4a6e8234e"
I20260929 16:08:36.805569 4776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a4a389e6b410b9709e85820e1af70" candidate_uuid: "2a7c348c04374ec39a644c8cf268ead6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8eca9e92a4d4b8b8f141fd623960677"
I20260929 16:08:36.805781 4908 raft_consensus.cc:3037] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:08:36.805781 4776 raft_consensus.cc:3037] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:08:36.809445 4908 raft_consensus.cc:2454] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a7c348c04374ec39a644c8cf268ead6 in term 1.
I20260929 16:08:36.809450 4776 raft_consensus.cc:2454] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a7c348c04374ec39a644c8cf268ead6 in term 1.
I20260929 16:08:36.810117 4974 leader_election.cc:302] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a7c348c04374ec39a644c8cf268ead6, 6969c39eddcc48edacc462a4a6e8234e; no voters:
I20260929 16:08:36.810468 5107 raft_consensus.cc:2781] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:08:36.810978 5107 raft_consensus.cc:686] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 1 LEADER]: Becoming Leader. State: Replica: 2a7c348c04374ec39a644c8cf268ead6, State: Running, Role: LEADER
I20260929 16:08:36.811636 5107 consensus_queue.cc:237] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:36.816515 4635 catalog_manager.cc:5495] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "2a7c348c04374ec39a644c8cf268ead6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } health_report { overall_health: UNKNOWN } } }
I20260929 16:08:36.884847 5095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.878646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36388 (local address 127.20.121.131:40311)
0929 16:08:36.878879 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.878886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.878908 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:08:36.879207 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.879212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.879299 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.879502 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:08:36.879885 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.880669 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.882506 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.882968 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.883123 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.883678 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:08:36.884206 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:08:36.884380 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:36.884464 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 16:08:36.884553 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 16:08:36.893997 4776 raft_consensus.cc:1261] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Refusing update from remote peer 2a7c348c04374ec39a644c8cf268ead6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:08:36.893965 4908 raft_consensus.cc:1261] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e [term 1 FOLLOWER]: Refusing update from remote peer 2a7c348c04374ec39a644c8cf268ead6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:08:36.895211 5110 consensus_queue.cc:1035] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:08:36.895584 5107 consensus_queue.cc:1035] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:08:36.912374 5113 mvcc.cc:205] Tried to move back new op lower bound from 7334699486786289664 to 7334699486464495616. Current Snapshot: MvccSnapshot[applied={T|T < 7334699486786289664}]
I20260929 16:08:36.929423 4636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:08:36.929740 4636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:08:36.933631 4636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Sending DeleteTablet for 3 replicas of tablet f64a4a389e6b410b9709e85820e1af70
I20260929 16:08:36.934872 4888 tablet_service.cc:1514] Processing DeleteTablet for tablet f64a4a389e6b410b9709e85820e1af70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:08:36 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:33538
I20260929 16:08:36.934942 5019 tablet_service.cc:1514] Processing DeleteTablet for tablet f64a4a389e6b410b9709e85820e1af70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:08:36 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:36374
I20260929 16:08:36.935833 5122 tablet_replica.cc:330] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: stopping tablet replica
I20260929 16:08:36.936331 5122 raft_consensus.cc:2229] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:08:36.936473 4756 tablet_service.cc:1514] Processing DeleteTablet for tablet f64a4a389e6b410b9709e85820e1af70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:08:36 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:36622
I20260929 16:08:36.937028 5122 raft_consensus.cc:2258] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:08:36.939647 5122 ts_tablet_manager.cc:1905] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:08:36.939711 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 4673
I20260929 16:08:36.947706 5122 ts_tablet_manager.cc:1918] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:08:36.948066 5122 log.cc:1196] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/f64a4a389e6b410b9709e85820e1af70
I20260929 16:08:36.948798 5122 ts_tablet_manager.cc:1939] T f64a4a389e6b410b9709e85820e1af70 P 6969c39eddcc48edacc462a4a6e8234e: Deleting consensus metadata
I20260929 16:08:36.951066 4622 catalog_manager.cc:4841] TS 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925): tablet f64a4a389e6b410b9709e85820e1af70 (table test-table [id=1de8cbfb233a46cf891c6bd00bb3bb78]) successfully deleted
I20260929 16:08:36.960772 5124 tablet_replica.cc:330] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: stopping tablet replica
I20260929 16:08:36.961725 5124 raft_consensus.cc:2229] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:08:36.962438 5124 raft_consensus.cc:2258] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:08:36.964295 5099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:36.941213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36630 (local address 127.20.121.129:43085)
0929 16:08:36.941403 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:36.941409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:36.941427 (+ 18us) server_negotiation.cc:407] Connection header received
0929 16:08:36.941491 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:36.941498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:36.941576 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:36.941674 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:36.943833 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.945002 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.947040 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:36.947722 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:36.947859 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:36.953561 (+ 5702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:36.953588 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:36.953616 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:36.953706 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:36.959044 (+ 5338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:36.960427 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:36.960438 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:36.960447 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:36.960662 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:36.961269 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:36.961274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:36.961277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:36.961999 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:36.963479 (+ 1480us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:36.963641 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 16:08:36.963901 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 16:08:36.965206 4816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:36.969236 5123 tablet_replica.cc:330] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: stopping tablet replica
I20260929 16:08:36.969904 5123 raft_consensus.cc:2229] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:08:36.970556 5123 raft_consensus.cc:2258] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:08:36.983382 5123 ts_tablet_manager.cc:1905] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:08:36.991813 5123 ts_tablet_manager.cc:1918] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:08:36.992017 5123 log.cc:1196] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/f64a4a389e6b410b9709e85820e1af70
I20260929 16:08:36.992606 5123 ts_tablet_manager.cc:1939] T f64a4a389e6b410b9709e85820e1af70 P 2a7c348c04374ec39a644c8cf268ead6: Deleting consensus metadata
I20260929 16:08:36.994776 4621 catalog_manager.cc:4841] TS 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311): tablet f64a4a389e6b410b9709e85820e1af70 (table test-table [id=1de8cbfb233a46cf891c6bd00bb3bb78]) successfully deleted
I20260929 16:08:37.064728 5124 ts_tablet_manager.cc:1905] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:08:37.065640 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 4673
W20260929 16:08:37.083508 4622 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.121.129:43085 (error 108)
I20260929 16:08:37.083784 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 4825
I20260929 16:08:37.090698 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:37.086711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.0.0.1:36646)
0929 16:08:37.090307 (+ 3596us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:37.090467 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3437,"thread_start_us":3210,"threads_started":1}
W20260929 16:08:37.091058 4622 catalog_manager.cc:4584] TS a8eca9e92a4d4b8b8f141fd623960677 (127.20.121.129:43085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f64a4a389e6b410b9709e85820e1af70: Network error: Client connection negotiation failed: client connection to 127.20.121.129:43085: connect: Connection refused (error 111)
I20260929 16:08:37.100469 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:37.084782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33540 (local address 127.20.121.130:44925)
0929 16:08:37.085369 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:37.085376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:37.085397 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:08:37.085467 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:37.085473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:37.085561 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:37.085678 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:37.088391 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:37.089636 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:37.091406 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:37.091820 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:37.091976 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:37.094590 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:37.094604 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:37.094620 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:37.094658 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:37.097451 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:37.098323 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:37.098331 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:37.098333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:37.098469 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:37.098870 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:37.098879 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:37.098881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:37.099250 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:37.099575 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:37.099882 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 16:08:37.100111 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428}
I20260929 16:08:37.101126 4948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:37.144706 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:37.144151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.0.0.1:36660)
0929 16:08:37.144388 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:37.144506 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260929 16:08:37.146085 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:37.145560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.0.0.1:36672)
0929 16:08:37.145767 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:37.145868 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 16:08:37.207064 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:37.206458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.0.0.1:36678)
0929 16:08:37.206712 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:37.206849 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260929 16:08:37.208401 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:37.207903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.0.0.1:36688)
0929 16:08:37.208147 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:37.208237 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260929 16:08:37.208964 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 4825
I20260929 16:08:37.227012 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 4956
I20260929 16:08:37.244463 5095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:37.227901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36398 (local address 127.20.121.131:40311)
0929 16:08:37.228123 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:37.228129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:37.228154 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:08:37.228269 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:37.228275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:37.228363 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:37.228496 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:37.229859 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:37.231152 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:37.232663 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:37.233401 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:37.233539 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:37.236304 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:37.236318 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:37.236341 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:37.236393 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:37.240749 (+ 4356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:37.241656 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:37.241668 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:37.241671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:37.241892 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:37.242510 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:37.242516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:37.242519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:37.243022 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:37.243377 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:37.243785 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 16:08:37.244035 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 16:08:37.245247 5079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:37.289112 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:37.288533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.0.0.1:36690)
0929 16:08:37.288808 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:37.288924 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260929 16:08:37.290369 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:37.289856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.0.0.1:36704)
0929 16:08:37.290121 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:37.290207 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260929 16:08:37.368778 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 4956
I20260929 16:08:37.388886 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 4603
I20260929 16:08:37.390254 4666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:08:37.505800 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 4603
I20260929 16:08:37.505861 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:37.505273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:43085 (local address 127.0.0.1:36714)
0929 16:08:37.505585 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:37.505703 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260929 16:08:57.528718 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34841
--webserver_interface=127.20.121.190
--webserver_port=34475
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:34841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:08:57.900524 5131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:57.900853 5131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:57.901023 5131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:57.908818 5131 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 16:08:57.908950 5131 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 16:08:57.909024 5131 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 16:08:57.909103 5131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:08:57.909196 5131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:57.909319 5131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:57.909384 5131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:08:57.909467 5131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:08:57.920281 5131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:34841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=34475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:57.922318 5131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:57.923970 5131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:57.936944 5137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:57.936972 5139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:57.937060 5136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:57.937909 5131 server_base.cc:1034] running on GCE node
I20260929 16:08:57.938884 5131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:57.940586 5131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:57.941910 5131 hybrid_clock.cc:648] HybridClock initialized: now 1790698137941831 us; error 35 us; skew 500 ppm
I20260929 16:08:57.942443 5131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:57 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698137, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:57.946085 5131 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:08:57.947749 5131 webserver.cc:460] Webserver started at http://127.20.121.190:34475/ using document root <none> and password file <none>
I20260929 16:08:57.948683 5131 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:57.948930 5131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:57.955296 5131 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 16:08:57.958714 5146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:57.960239 5131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:08:57.960470 5131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "a592aebe041c41dab4a43d091e589bc8"
format_stamp: "Formatted at 2026-09-29 16:08:11 on dist-test-slave-x07d"
server_key: "e12a0a0e111b30a916ac8a7bad917f46"
server_key_iv: "8df6923090f8a165ee4c9aa9027e6d25"
server_key_version: "encryptionkey@0"
I20260929 16:08:57.961580 5131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:57.981177 5131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:57.985981 5131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:57.986346 5131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:58.006358 5131 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:34841
I20260929 16:08:58.006402 5197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:34841 every 8 connection(s)
I20260929 16:08:58.007962 5131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:08:58.015547 5198 sys_catalog.cc:263] Verifying existing consensus state
I20260929 16:08:58.016646 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 5131
I20260929 16:08:58.017805 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:43085
--local_ip_for_outbound_sockets=127.20.121.129
--tserver_master_addrs=127.20.121.190:34841
--webserver_port=46487
--webserver_interface=127.20.121.129
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:08:58.020962 5198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Bootstrap starting.
I20260929 16:08:58.043838 5198 log.cc:824] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Log is configured to *not* fsync() on all Append() calls
I20260929 16:08:58.060904 5198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 16:08:58.061558 5198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Bootstrap complete.
I20260929 16:08:58.071350 5198 raft_consensus.cc:348] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:58.072288 5198 raft_consensus.cc:729] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a592aebe041c41dab4a43d091e589bc8, State: Initialized, Role: FOLLOWER
I20260929 16:08:58.073038 5198 consensus_queue.cc:260] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:58.073295 5198 raft_consensus.cc:388] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:08:58.073429 5198 raft_consensus.cc:482] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:08:58.073628 5198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 2 FOLLOWER]: Advancing to term 3
I20260929 16:08:58.077925 5198 raft_consensus.cc:504] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:58.078614 5198 leader_election.cc:302] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a592aebe041c41dab4a43d091e589bc8; no voters:
I20260929 16:08:58.079372 5198 leader_election.cc:288] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 16:08:58.079535 5202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 16:08:58.080663 5202 raft_consensus.cc:686] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [term 3 LEADER]: Becoming Leader. State: Replica: a592aebe041c41dab4a43d091e589bc8, State: Running, Role: LEADER
I20260929 16:08:58.081367 5202 consensus_queue.cc:237] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } }
I20260929 16:08:58.081733 5198 sys_catalog.cc:564] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:08:58.083986 5204 sys_catalog.cc:455] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a592aebe041c41dab4a43d091e589bc8. Latest consensus state: current_term: 3 leader_uuid: "a592aebe041c41dab4a43d091e589bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } } }
I20260929 16:08:58.084394 5204 sys_catalog.cc:458] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:08:58.085194 5203 sys_catalog.cc:455] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a592aebe041c41dab4a43d091e589bc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a592aebe041c41dab4a43d091e589bc8" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34841 } } }
I20260929 16:08:58.085937 5203 sys_catalog.cc:458] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:08:58.091668 5207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:08:58.104161 5207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:08:58.106964 5207 catalog_manager.cc:1259] Loaded cluster ID: 37a8b4aa9b0d4cfaa92076c0cacc9989
I20260929 16:08:58.107230 5207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:08:58.113925 5207 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:08:58.116752 5207 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Loaded TSK: 0
I20260929 16:08:58.117129 5207 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a592aebe041c41dab4a43d091e589bc8: Loaded TSK: 1
F20260929 16:08:58.118933 5207 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe3fcfec9ad google::LogMessage::Fail() at ??:0
@ 0x7fe3fcfef88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe3fcfec4de google::LogMessage::Flush() at ??:0
@ 0x7fe3fcfeda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe400d507d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe400d3f73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe412a1b1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe412a1a783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
I20260929 16:08:58.154819 5220 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fe412aa62a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fe412aa60c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fe412af05e9 std::function<>::operator()() at ??:0
@ 0x7fe412a1ecdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fe412a1cc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe412aa4610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fe412aa4439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe3ffc38345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe3ffc3b91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe3ffc3b799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe3ffc16eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe402ef06db start_thread at ??:0
@ 0x7fe3fbe8661f clone at ??:0
W20260929 16:08:58.424324 5200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:58.424710 5200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:58.424888 5200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:58.432866 5200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:58.433058 5200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:58.433248 5200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:08:58.444257 5200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:43085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=46487
--tserver_master_addrs=127.20.121.190:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:58.445760 5200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:58.447284 5200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:58.462715 5245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:58.462751 5246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:58.462715 5248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:58.463653 5200 server_base.cc:1034] running on GCE node
I20260929 16:08:58.464380 5200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:58.465528 5200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:58.466784 5200 hybrid_clock.cc:648] HybridClock initialized: now 1790698138466721 us; error 56 us; skew 500 ppm
I20260929 16:08:58.467175 5200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:58 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:58.470909 5200 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:08:58.472698 5200 webserver.cc:460] Webserver started at http://127.20.121.129:46487/ using document root <none> and password file <none>
I20260929 16:08:58.473512 5200 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:58.473662 5200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:58.480154 5200 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 16:08:58.484066 5255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:58.486101 5200 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260929 16:08:58.486340 5200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "a8eca9e92a4d4b8b8f141fd623960677"
format_stamp: "Formatted at 2026-09-29 16:08:12 on dist-test-slave-x07d"
server_key: "6a0f05cc983c9f9fcf6251a26cc80cc3"
server_key_iv: "ce34475799aee60c9b84f1e7bc4b98af"
server_key_version: "encryptionkey@0"
I20260929 16:08:58.487448 5200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:58.515422 5200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:58.520818 5200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:58.521162 5200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:58.522521 5200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:08:58.524601 5262 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260929 16:08:58.531328 5200 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 16:08:58.531481 5200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260929 16:08:58.531683 5200 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 16:08:58.535269 5200 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 16:08:58.535386 5200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260929 16:08:58.535892 5262 tablet_bootstrap.cc:492] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Bootstrap starting.
I20260929 16:08:58.545612 5262 log.cc:824] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Log is configured to *not* fsync() on all Append() calls
I20260929 16:08:58.563289 5262 tablet_bootstrap.cc:492] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 16:08:58.563979 5262 tablet_bootstrap.cc:492] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Bootstrap complete.
I20260929 16:08:58.564496 5262 ts_tablet_manager.cc:1397] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Time spent bootstrapping tablet: real 0.029s user 0.021s sys 0.004s
I20260929 16:08:58.575151 5262 raft_consensus.cc:348] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:58.577051 5262 raft_consensus.cc:729] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8eca9e92a4d4b8b8f141fd623960677, State: Initialized, Role: FOLLOWER
I20260929 16:08:58.577857 5262 consensus_queue.cc:260] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:58.580395 5262 ts_tablet_manager.cc:1428] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260929 16:08:58.582218 5200 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:43085
I20260929 16:08:58.582263 5370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:43085 every 8 connection(s)
I20260929 16:08:58.583633 5200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:08:58.589299 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:58.587072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:58767)
0929 16:08:58.588008 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:58.588312 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":414,"threads_started":1}
W20260929 16:08:58.590109 5371 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:34841 (0 consecutive failures): Network error: Failed to ping master at 127.20.121.190:34841: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
I20260929 16:08:58.591130 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:58.590749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:59157)
0929 16:08:58.590926 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:58.591012 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 16:08:58.592136 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:58.591707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:52933)
0929 16:08:58.591898 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:58.591981 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 16:08:58.592412 5371 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 16:08:58.593247 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 5200
I20260929 16:08:58.593894 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:44925
--local_ip_for_outbound_sockets=127.20.121.130
--tserver_master_addrs=127.20.121.190:34841
--webserver_port=39691
--webserver_interface=127.20.121.130
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:08:58.950209 5374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:58.950510 5374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:58.950673 5374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:58.958272 5374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:58.958405 5374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:58.958582 5374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:08:58.968804 5374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:44925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=39691
--tserver_master_addrs=127.20.121.190:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:58.970300 5374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:58.971709 5374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:58.988047 5379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:58.987996 5380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:58.988026 5382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:58.988905 5374 server_base.cc:1034] running on GCE node
I20260929 16:08:58.989712 5374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:58.990880 5374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:58.992170 5374 hybrid_clock.cc:648] HybridClock initialized: now 1790698138992087 us; error 51 us; skew 500 ppm
I20260929 16:08:58.992619 5374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:58 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:58.996012 5374 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:08:58.997648 5374 webserver.cc:460] Webserver started at http://127.20.121.130:39691/ using document root <none> and password file <none>
I20260929 16:08:58.998483 5374 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:58.998593 5374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:59.004860 5374 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 16:08:59.008505 5389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:59.010385 5374 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:08:59.010607 5374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "6969c39eddcc48edacc462a4a6e8234e"
format_stamp: "Formatted at 2026-09-29 16:08:12 on dist-test-slave-x07d"
server_key: "c421d90491b1660039cb6c43432f8907"
server_key_iv: "49cea2febfc0ae6c6d2ecddab4d002a1"
server_key_version: "encryptionkey@0"
I20260929 16:08:59.011745 5374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:59.052340 5374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:59.058195 5374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:59.058521 5374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:59.059681 5374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:08:59.061196 5374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:08:59.061295 5374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:59.061429 5374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:08:59.061522 5374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:59.097082 5374 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:44925
I20260929 16:08:59.097249 5501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:44925 every 8 connection(s)
I20260929 16:08:59.098699 5374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:08:59.101962 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 5374
I20260929 16:08:59.102617 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:40311
--local_ip_for_outbound_sockets=127.20.121.131
--tserver_master_addrs=127.20.121.190:34841
--webserver_port=46385
--webserver_interface=127.20.121.131
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:08:59.104080 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.101935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:38045)
0929 16:08:59.102891 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:59.103205 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":369,"threads_started":1}
W20260929 16:08:59.104671 5502 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:34841 (0 consecutive failures): Network error: Failed to ping master at 127.20.121.190:34841: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
I20260929 16:08:59.105513 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.105163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:57967)
0929 16:08:59.105320 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:59.105404 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 16:08:59.106374 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.106069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:51647)
0929 16:08:59.106188 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:59.106269 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260929 16:08:59.106575 5502 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 16:08:59.444310 5505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:08:59.444684 5505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:08:59.444857 5505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:08:59.452747 5505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:08:59.452939 5505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:08:59.453114 5505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:08:59.463728 5505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:40311
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=46385
--tserver_master_addrs=127.20.121.190:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:08:59.465193 5505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:08:59.466627 5505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:08:59.481985 5511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:59.481956 5510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:08:59.481920 5513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:08:59.484167 5505 server_base.cc:1034] running on GCE node
I20260929 16:08:59.484989 5505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:08:59.486609 5505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:08:59.488023 5505 hybrid_clock.cc:648] HybridClock initialized: now 1790698139487942 us; error 73 us; skew 500 ppm
I20260929 16:08:59.488514 5505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:08:59 dist-test-slave-x07d krb5kdc[4051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698139, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:08:59.492254 5505 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:08:59.494378 5505 webserver.cc:460] Webserver started at http://127.20.121.131:46385/ using document root <none> and password file <none>
I20260929 16:08:59.495105 5505 fs_manager.cc:359] Metadata directory not provided
I20260929 16:08:59.495222 5505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:08:59.501546 5505 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 16:08:59.504776 5520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:59.506624 5505 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:08:59.506786 5505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "2a7c348c04374ec39a644c8cf268ead6"
format_stamp: "Formatted at 2026-09-29 16:08:13 on dist-test-slave-x07d"
server_key: "b91de395e758016fc1a0b7d9e18e55c3"
server_key_iv: "f1bb16cf4e90cef23afffa4844c8e323"
server_key_version: "encryptionkey@0"
I20260929 16:08:59.507880 5505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:08:59.530448 5505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:08:59.534931 5505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:08:59.535229 5505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:08:59.536312 5505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:08:59.537766 5505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:08:59.537863 5505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:08:59.538048 5505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:08:59.538152 5505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260929 16:08:59.574216 5505 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:40311
I20260929 16:08:59.574287 5632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:40311 every 8 connection(s)
I20260929 16:08:59.575675 5505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:08:59.580627 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 5505
I20260929 16:08:59.581367 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.578816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:40083)
0929 16:08:59.579667 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:59.580009 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":312,"threads_started":1}
W20260929 16:08:59.582705 5633 heartbeater.cc:644] Failed to heartbeat to 127.20.121.190:34841 (0 consecutive failures): Network error: Failed to ping master at 127.20.121.190:34841: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
I20260929 16:08:59.583745 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.583337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:34541)
0929 16:08:59.583519 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:59.583609 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 16:08:59.584767 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.584383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:34869)
0929 16:08:59.584533 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:59.584624 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 16:08:59.585031 5633 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 16:08:59.594117 5637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.593140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:46695)
0929 16:08:59.593716 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:59.593844 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":276,"threads_started":1}
I20260929 16:08:59.885135 5638 raft_consensus.cc:482] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:08:59.885481 5638 raft_consensus.cc:504] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:08:59.887279 5638 leader_election.cc:288] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311), 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925)
Sep 29 16:08:59 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
Sep 29 16:08:59 dist-test-slave-x07d krb5kdc[4051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698138, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:08:59.914695 5637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.887618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:40311 (local address 127.20.121.129:58847)
0929 16:08:59.888053 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:59.888167 (+ 114us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:59.888951 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:59.894153 (+ 5202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:59.894165 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:59.895100 (+ 935us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:08:59.895888 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:59.895912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:59.897632 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:59.897636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:59.899179 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:59.899198 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:59.900299 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:59.900307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:59.900599 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:59.901538 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:08:59.901557 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:08:59.904846 (+ 3289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:08:59.909660 (+ 4814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:59.909672 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:59.909691 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:59.910662 (+ 971us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:59.911272 (+ 610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:59.911278 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:59.911282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:59.911581 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:59.912674 (+ 1093us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:08:59.912681 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:08:59.913167 (+ 486us) client_negotiation.cc:769] Sending connection context
0929 16:08:59.913497 (+ 330us) client_negotiation.cc:240] Negotiation successful
0929 16:08:59.914267 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":338,"spinlock_wait_cycles":11904}
I20260929 16:08:59.914711 5640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.887874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:58847 (local address 127.20.121.131:40311)
0929 16:08:59.890737 (+ 2863us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:59.890760 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:59.890836 (+ 76us) server_negotiation.cc:407] Connection header received
0929 16:08:59.893112 (+ 2276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:59.893128 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:59.893580 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:59.893872 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:59.896124 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:59.897378 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:59.899394 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:59.900062 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:59.900244 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:59.905171 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:59.905184 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:59.905206 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:59.905260 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:59.909431 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:59.910920 (+ 1489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:59.910930 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:59.910939 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:59.911136 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:59.911718 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:59.911723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:59.911727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:59.912513 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:59.912714 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:59.913808 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0929 16:08:59.914298 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2588,"thread_start_us":2367,"threads_started":1}
I20260929 16:08:59.915079 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.887777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:44925 (local address 127.20.121.129:33091)
0929 16:08:59.889969 (+ 2192us) negotiation.cc:107] Waiting for socket to connect
0929 16:08:59.889997 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 16:08:59.890184 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:08:59.894383 (+ 4199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:08:59.894388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:08:59.895098 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:08:59.895888 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:59.895912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:59.897625 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:59.897630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:59.899183 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:08:59.899197 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:08:59.900354 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:08:59.900361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:08:59.900594 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:59.901484 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:08:59.901507 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:08:59.904731 (+ 3224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:08:59.909660 (+ 4929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:59.909672 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:59.909691 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:59.910662 (+ 971us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:59.911272 (+ 610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:08:59.911278 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:08:59.911282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:08:59.911581 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:08:59.912774 (+ 1193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:08:59.912779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:08:59.913151 (+ 372us) client_negotiation.cc:769] Sending connection context
0929 16:08:59.913497 (+ 346us) client_negotiation.cc:240] Negotiation successful
0929 16:08:59.914159 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1983,"mutex_wait_us":220,"spinlock_wait_cycles":71808,"thread_start_us":1734,"threads_started":1}
I20260929 16:08:59.916069 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:08:59.887982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:33091 (local address 127.20.121.130:44925)
0929 16:08:59.891480 (+ 3498us) server_negotiation.cc:206] Beginning negotiation
0929 16:08:59.891497 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 16:08:59.891548 (+ 51us) server_negotiation.cc:407] Connection header received
0929 16:08:59.893571 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:08:59.893582 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:08:59.893999 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:08:59.894409 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:08:59.896118 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:59.897394 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:59.899348 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:08:59.900085 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:08:59.900331 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:08:59.905177 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:08:59.905186 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:08:59.905206 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:08:59.905261 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:08:59.909436 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:59.910851 (+ 1415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:59.910863 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:59.910874 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:59.911124 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:08:59.911718 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:08:59.911723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:08:59.911727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:08:59.912496 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:08:59.912852 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0929 16:08:59.915211 (+ 2359us) server_negotiation.cc:299] Negotiation successful
0929 16:08:59.915658 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3294,"thread_start_us":3063,"threads_started":1}
W20260929 16:08:59.918567 5257 leader_election.cc:341] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311): Not found: Tablet not found: f64a4a389e6b410b9709e85820e1af70
W20260929 16:08:59.919174 5258 leader_election.cc:341] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925): Not found: Tablet not found: f64a4a389e6b410b9709e85820e1af70
I20260929 16:08:59.919319 5258 leader_election.cc:302] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8eca9e92a4d4b8b8f141fd623960677; no voters: 2a7c348c04374ec39a644c8cf268ead6, 6969c39eddcc48edacc462a4a6e8234e
I20260929 16:08:59.919800 5638 raft_consensus.cc:2726] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 16:09:00.108196 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:00.107198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:59329)
0929 16:09:00.107711 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:00.107924 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":244,"threads_started":1}
I20260929 16:09:00.586745 5643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:00.585722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:59477)
0929 16:09:00.586404 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:00.586538 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":307,"threads_started":1}
I20260929 16:09:00.595891 5644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:00.595144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:60233)
0929 16:09:00.595617 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:00.595711 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":239,"threads_started":1}
I20260929 16:09:01.110080 5645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:01.109221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:40423)
0929 16:09:01.109673 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:01.109838 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":235,"threads_started":1}
I20260929 16:09:01.588675 5646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:01.587865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:36709)
0929 16:09:01.588389 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:01.588484 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":287,"threads_started":1}
I20260929 16:09:01.597860 5647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:01.597002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:37287)
0929 16:09:01.597588 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:01.597675 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":326,"threads_started":1}
I20260929 16:09:01.890460 5648 raft_consensus.cc:482] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:01.890723 5648 raft_consensus.cc:504] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:09:01.891453 5648 leader_election.cc:288] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311), 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925)
W20260929 16:09:01.892772 5257 leader_election.cc:341] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311): Not found: Tablet not found: f64a4a389e6b410b9709e85820e1af70
W20260929 16:09:01.892974 5258 leader_election.cc:341] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925): Not found: Tablet not found: f64a4a389e6b410b9709e85820e1af70
I20260929 16:09:01.893115 5258 leader_election.cc:302] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8eca9e92a4d4b8b8f141fd623960677; no voters: 2a7c348c04374ec39a644c8cf268ead6, 6969c39eddcc48edacc462a4a6e8234e
I20260929 16:09:01.893373 5648 raft_consensus.cc:2726] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 16:09:02.111968 5649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:02.111079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:56775)
0929 16:09:02.111649 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:02.111743 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":271,"threads_started":1}
I20260929 16:09:02.590721 5650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:02.589886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:60741)
0929 16:09:02.590442 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:02.590556 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":303,"threads_started":1}
I20260929 16:09:02.599617 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:02.598906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:55179)
0929 16:09:02.599353 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:02.599455 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":232,"threads_started":1}
I20260929 16:09:03.113930 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:03.113041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:49779)
0929 16:09:03.113578 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:03.113671 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":287,"threads_started":1}
I20260929 16:09:03.592782 5653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:03.591903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:35883)
0929 16:09:03.592468 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:03.592571 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":294,"threads_started":1}
I20260929 16:09:03.601691 5654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:03.600760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:48725)
0929 16:09:03.601401 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:03.601499 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":347,"threads_started":1}
I20260929 16:09:04.115999 5655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:04.115037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:56563)
0929 16:09:04.115596 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:04.115810 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":280,"threads_started":1}
I20260929 16:09:04.594750 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:04.593952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:60897)
0929 16:09:04.594486 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:04.594585 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":239,"threads_started":1}
I20260929 16:09:04.603686 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:04.602933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:41037)
0929 16:09:04.603408 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:04.603521 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":240,"threads_started":1}
I20260929 16:09:04.720193 5658 raft_consensus.cc:482] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:04.720477 5658 raft_consensus.cc:504] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:09:04.721182 5658 leader_election.cc:288] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311), 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925)
W20260929 16:09:04.722373 5257 leader_election.cc:341] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311): Not found: Tablet not found: f64a4a389e6b410b9709e85820e1af70
W20260929 16:09:04.722611 5258 leader_election.cc:341] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925): Not found: Tablet not found: f64a4a389e6b410b9709e85820e1af70
I20260929 16:09:04.722807 5258 leader_election.cc:302] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8eca9e92a4d4b8b8f141fd623960677; no voters: 2a7c348c04374ec39a644c8cf268ead6, 6969c39eddcc48edacc462a4a6e8234e
I20260929 16:09:04.723140 5658 raft_consensus.cc:2726] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 16:09:05.118212 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:05.117029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:54305)
0929 16:09:05.117819 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:05.117929 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":505,"threads_started":1}
I20260929 16:09:05.596889 5660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:05.595989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:48187)
0929 16:09:05.596598 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:05.596692 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":347,"threads_started":1}
I20260929 16:09:05.605993 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:05.605034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:36437)
0929 16:09:05.605626 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:05.605729 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":253,"threads_started":1}
I20260929 16:09:06.120329 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:06.119342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:58335)
0929 16:09:06.120003 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:06.120108 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":270,"threads_started":1}
I20260929 16:09:06.599083 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:06.597991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:39709)
0929 16:09:06.598713 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:06.598807 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":296,"threads_started":1}
I20260929 16:09:06.607996 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:06.607146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:47207)
0929 16:09:06.607742 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:06.607830 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":351,"threads_started":1}
I20260929 16:09:07.122300 5665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:07.121485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:37261)
0929 16:09:07.121964 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:07.122115 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":237,"threads_started":1}
I20260929 16:09:07.601042 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:07.600143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:54791)
0929 16:09:07.600669 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:07.600779 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":239,"threads_started":1}
I20260929 16:09:07.609829 5667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:07.608980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:51755)
0929 16:09:07.609530 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:07.609625 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":294,"threads_started":1}
I20260929 16:09:08.073982 5668 raft_consensus.cc:482] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:08.074317 5668 raft_consensus.cc:504] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:09:08.075153 5668 leader_election.cc:288] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311), 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925)
W20260929 16:09:08.076200 5258 leader_election.cc:341] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925): Not found: Tablet not found: f64a4a389e6b410b9709e85820e1af70
W20260929 16:09:08.076412 5257 leader_election.cc:341] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311): Not found: Tablet not found: f64a4a389e6b410b9709e85820e1af70
I20260929 16:09:08.076597 5257 leader_election.cc:302] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8eca9e92a4d4b8b8f141fd623960677; no voters: 2a7c348c04374ec39a644c8cf268ead6, 6969c39eddcc48edacc462a4a6e8234e
I20260929 16:09:08.076967 5668 raft_consensus.cc:2726] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 16:09:08.123996 5669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:08.123167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:60609)
0929 16:09:08.123718 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:08.123838 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":270,"threads_started":1}
I20260929 16:09:08.603055 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:08.602129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:33989)
0929 16:09:08.602775 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:08.602877 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":332,"threads_started":1}
I20260929 16:09:08.611804 5671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:08.610904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:46639)
0929 16:09:08.611438 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:08.611608 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":254,"threads_started":1}
I20260929 16:09:09.125751 5672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:09.125024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:46217)
0929 16:09:09.125488 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:09.125572 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":266,"threads_started":1}
I20260929 16:09:09.605175 5673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:09.604296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:34461)
0929 16:09:09.604884 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:09.604976 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":349,"threads_started":1}
I20260929 16:09:09.613864 5674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:09.613066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:44605)
0929 16:09:09.613584 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:09.613686 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":275,"threads_started":1}
I20260929 16:09:10.127673 5675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:10.126857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:42807)
0929 16:09:10.127389 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:10.127503 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":247,"threads_started":1}
I20260929 16:09:10.607299 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:10.606424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:54721)
0929 16:09:10.607030 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:10.607130 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":268,"threads_started":1}
I20260929 16:09:10.616004 5677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:10.615034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:43479)
0929 16:09:10.615650 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:10.615751 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":249,"threads_started":1}
I20260929 16:09:11.129791 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:11.128935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:38539)
0929 16:09:11.129493 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:11.129593 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":274,"threads_started":1}
I20260929 16:09:11.609421 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:11.608580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:54339)
0929 16:09:11.609152 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:11.609249 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":304,"threads_started":1}
I20260929 16:09:11.618809 5680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:11.617607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:50503)
0929 16:09:11.618495 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:11.618618 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":450,"threads_started":1}
I20260929 16:09:12.131898 5681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:12.131059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:34443)
0929 16:09:12.131600 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:12.131718 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":270,"threads_started":1}
I20260929 16:09:12.611917 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:12.610831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:33839)
0929 16:09:12.611482 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:12.611631 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":357,"threads_started":1}
I20260929 16:09:12.621032 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:12.620183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:60085)
0929 16:09:12.620681 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:12.620793 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":262,"threads_started":1}
I20260929 16:09:13.133997 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:13.133098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:46369)
0929 16:09:13.133679 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:13.133790 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":296,"threads_started":1}
I20260929 16:09:13.614336 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:13.613289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:55551)
0929 16:09:13.613831 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:13.613935 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":275,"threads_started":1}
I20260929 16:09:13.623545 5686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:13.622677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.129:49431)
0929 16:09:13.623273 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:13.623370 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":277,"threads_started":1}
I20260929 16:09:14.136387 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:14.135382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.130:35297)
0929 16:09:14.135983 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:14.136146 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":279,"threads_started":1}
I20260929 16:09:14.578599 5688 raft_consensus.cc:482] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:14.578864 5688 raft_consensus.cc:504] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a7c348c04374ec39a644c8cf268ead6" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 40311 } } peers { permanent_uuid: "6969c39eddcc48edacc462a4a6e8234e" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44925 } } peers { permanent_uuid: "a8eca9e92a4d4b8b8f141fd623960677" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 43085 } }
I20260929 16:09:14.579630 5688 leader_election.cc:288] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311), 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925)
W20260929 16:09:14.580713 5257 leader_election.cc:341] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a7c348c04374ec39a644c8cf268ead6 (127.20.121.131:40311): Not found: Tablet not found: f64a4a389e6b410b9709e85820e1af70
W20260929 16:09:14.580914 5258 leader_election.cc:341] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6969c39eddcc48edacc462a4a6e8234e (127.20.121.130:44925): Not found: Tablet not found: f64a4a389e6b410b9709e85820e1af70
I20260929 16:09:14.581089 5258 leader_election.cc:302] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a8eca9e92a4d4b8b8f141fd623960677; no voters: 2a7c348c04374ec39a644c8cf268ead6, 6969c39eddcc48edacc462a4a6e8234e
I20260929 16:09:14.581419 5688 raft_consensus.cc:2726] T f64a4a389e6b410b9709e85820e1af70 P a8eca9e92a4d4b8b8f141fd623960677 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 16:09:14.593138 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 5200
I20260929 16:09:14.611402 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:14.594534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.20.121.129:43085)
0929 16:09:14.595651 (+ 1117us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:14.595659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:14.595696 (+ 37us) server_negotiation.cc:407] Connection header received
0929 16:09:14.595766 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:14.595773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:14.596015 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:14.596162 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:14.597884 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:14.599017 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:14.600475 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:14.601148 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:14.601265 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:14.604391 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:14.604407 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:14.604422 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:14.604476 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:14.607912 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:14.608867 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:14.608876 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:14.608879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:14.609032 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:14.609467 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:14.609472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:14.609474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:14.609903 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:14.610245 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:14.610585 (+ 340us) server_negotiation.cc:299] Negotiation successful
0929 16:09:14.610813 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":753,"threads_started":1}
I20260929 16:09:14.613006 5366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:14.617292 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:14.615808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:34841 (local address 127.20.121.131:42395)
0929 16:09:14.616412 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:14.616597 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.190:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":338,"threads_started":1}
I20260929 16:09:14.706534 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 5200
I20260929 16:09:14.725876 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 5374
I20260929 16:09:14.739776 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:14.726720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57680 (local address 127.20.121.130:44925)
0929 16:09:14.727249 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:14.727253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:14.727270 (+ 17us) server_negotiation.cc:407] Connection header received
0929 16:09:14.727326 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:14.727329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:14.727391 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:14.727488 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:14.728891 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:14.729659 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:14.730963 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:14.731415 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:14.731525 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:14.734160 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:14.734166 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:14.734169 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:14.734206 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:14.736875 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:14.737547 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:14.737550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:14.737552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:14.737685 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:14.738473 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:14.738476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:14.738478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:14.738753 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:14.738959 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:14.739194 (+ 235us) server_negotiation.cc:299] Negotiation successful
0929 16:09:14.739368 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":322,"threads_started":1}
I20260929 16:09:14.740665 5497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:14.826812 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 5374
I20260929 16:09:14.844601 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 5505
I20260929 16:09:14.859452 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:14.845569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38726 (local address 127.20.121.131:40311)
0929 16:09:14.846217 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:14.846221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:14.846237 (+ 16us) server_negotiation.cc:407] Connection header received
0929 16:09:14.846292 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:14.846295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:14.846361 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:14.846490 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:14.847852 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:14.848629 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:14.849859 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:14.850416 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:14.850563 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:14.853485 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:14.853490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:14.853494 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:14.853533 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:14.856409 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:14.857185 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:14.857191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:14.857196 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:14.857349 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:14.857829 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:14.857834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:14.857840 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:14.858187 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:14.858485 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:14.858793 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 16:09:14.859025 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":325,"threads_started":1}
I20260929 16:09:14.860291 5628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:14.945323 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 5505
2026-09-29T16:09:14Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65027 ms)
[----------] 30 tests from SecurityITest (313442 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:15 dist-test-slave-x07d krb5kdc[5699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:15 dist-test-slave-x07d krb5kdc[5699](info): set up 2 sockets
Sep 29 16:09:15 dist-test-slave-x07d krb5kdc[5699](info): commencing operation
krb5kdc: starting...
W20260929 16:09:16.066419 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:16 dist-test-slave-x07d krb5kdc[5699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:09:16.346599 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33233
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:33233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:16.692090 5712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:16.692394 5712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:16.692574 5712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:16.700161 5712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:16.700350 5712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:16.700476 5712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:16.700578 5712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:16.700664 5712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:16.711022 5712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:33233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:16.712597 5712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:16.714661 5712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:16.729561 5718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:16.729643 5720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:16.729592 5717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:16.730417 5712 server_base.cc:1034] running on GCE node
I20260929 16:09:16.731402 5712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:16.733126 5712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:16.734437 5712 hybrid_clock.cc:648] HybridClock initialized: now 1790698156734385 us; error 67 us; skew 500 ppm
I20260929 16:09:16.735003 5712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:16 dist-test-slave-x07d krb5kdc[5699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698156, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:16.738780 5712 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:09:16.740687 5712 webserver.cc:460] Webserver started at http://127.20.121.190:45645/ using document root <none> and password file <none>
I20260929 16:09:16.741591 5712 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:16.741765 5712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:16.742187 5712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:16.745563 5712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "3261065b215b4005936c920d24031f95"
format_stamp: "Formatted at 2026-09-29 16:09:16 on dist-test-slave-x07d"
server_key: "a7485a783ddc331f10f62af247cd8d89"
server_key_iv: "6a98e98b50a1a382e1885a91e48efcaf"
server_key_version: "encryptionkey@0"
I20260929 16:09:16.746574 5712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "3261065b215b4005936c920d24031f95"
format_stamp: "Formatted at 2026-09-29 16:09:16 on dist-test-slave-x07d"
server_key: "a7485a783ddc331f10f62af247cd8d89"
server_key_iv: "6a98e98b50a1a382e1885a91e48efcaf"
server_key_version: "encryptionkey@0"
I20260929 16:09:16.752784 5712 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260929 16:09:16.757423 5727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:16.759208 5712 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:09:16.759469 5712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "3261065b215b4005936c920d24031f95"
format_stamp: "Formatted at 2026-09-29 16:09:16 on dist-test-slave-x07d"
server_key: "a7485a783ddc331f10f62af247cd8d89"
server_key_iv: "6a98e98b50a1a382e1885a91e48efcaf"
server_key_version: "encryptionkey@0"
I20260929 16:09:16.759737 5712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:16.779686 5712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:16.784832 5712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:16.785204 5712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:16.806901 5712 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:33233
I20260929 16:09:16.806941 5778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:33233 every 8 connection(s)
I20260929 16:09:16.808557 5712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:16.812973 5779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:16.812968 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 5712
I20260929 16:09:16.813313 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:16.814128 20966 external_mini_cluster.cc:1428] Setting key 8d62705217f619353adc00d86de7a7a3
I20260929 16:09:16.830575 5779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95: Bootstrap starting.
Sep 29 16:09:16 dist-test-slave-x07d krb5kdc[5699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:16.834924 5779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:16.836239 5779 log.cc:824] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:16.839815 5779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95: No bootstrap required, opened a new log
I20260929 16:09:16.842140 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:16.816387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43054 (local address 127.20.121.190:33233)
0929 16:09:16.817376 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:16.817390 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:16.817442 (+ 52us) server_negotiation.cc:407] Connection header received
0929 16:09:16.820087 (+ 2645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:16.820119 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:16.820642 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:16.821059 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:16.823411 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:16.824929 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:16.827379 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:16.828315 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:16.828495 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:16.834147 (+ 5652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:16.834158 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:16.834174 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:16.834222 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:16.837585 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:16.838503 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:16.838510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:16.838517 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:16.838703 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:16.839143 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:16.839147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:16.839149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:16.839710 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:16.840038 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:16.840589 (+ 551us) server_negotiation.cc:299] Negotiation successful
0929 16:09:16.841031 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":486,"threads_started":1}
I20260929 16:09:16.850423 5779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3261065b215b4005936c920d24031f95" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33233 } }
I20260929 16:09:16.850893 5779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:16.851018 5779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3261065b215b4005936c920d24031f95, State: Initialized, Role: FOLLOWER
I20260929 16:09:16.851722 5779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3261065b215b4005936c920d24031f95" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33233 } }
I20260929 16:09:16.852033 5779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:16.852209 5779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:16.852414 5779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:16.855274 5779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3261065b215b4005936c920d24031f95" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33233 } }
I20260929 16:09:16.855922 5779 leader_election.cc:302] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3261065b215b4005936c920d24031f95; no voters:
I20260929 16:09:16.856590 5779 leader_election.cc:288] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:16.856770 5784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:16.857653 5784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [term 1 LEADER]: Becoming Leader. State: Replica: 3261065b215b4005936c920d24031f95, State: Running, Role: LEADER
I20260929 16:09:16.858408 5784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3261065b215b4005936c920d24031f95" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33233 } }
I20260929 16:09:16.859377 5779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:16.861948 5786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3261065b215b4005936c920d24031f95. Latest consensus state: current_term: 1 leader_uuid: "3261065b215b4005936c920d24031f95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3261065b215b4005936c920d24031f95" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33233 } } }
I20260929 16:09:16.861909 5785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3261065b215b4005936c920d24031f95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3261065b215b4005936c920d24031f95" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33233 } } }
I20260929 16:09:16.862505 5785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:16.863204 5786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:16.864720 5791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:16.874409 5791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:16.885773 5791 catalog_manager.cc:1347] Generated new cluster ID: 30a604ccd5804225b2d80b47b86749ed
I20260929 16:09:16.885933 5791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:16.902731 5791 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:16.905004 5791 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:16.915894 5791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95: Generated new TSK 0
I20260929 16:09:16.917076 5791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:09:17.006323 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33233
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:17.366535 5805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:17.366830 5805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:17.366938 5805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:17.374557 5805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:17.374691 5805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:17.374809 5805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:09:17.384938 5805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:17.386385 5805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:17.387804 5805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:17.402349 5813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:17.402460 5811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:17.402479 5810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:17.404100 5805 server_base.cc:1034] running on GCE node
I20260929 16:09:17.404970 5805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:17.406318 5805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:17.407603 5805 hybrid_clock.cc:648] HybridClock initialized: now 1790698157407549 us; error 78 us; skew 500 ppm
I20260929 16:09:17.408035 5805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:17 dist-test-slave-x07d krb5kdc[5699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:17.412201 5805 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:09:17.414359 5805 webserver.cc:460] Webserver started at http://127.20.121.129:35355/ using document root <none> and password file <none>
I20260929 16:09:17.415212 5805 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:17.415401 5805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:17.415784 5805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:17.419842 5805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "4f411e2e371c4fb9b7655e8ec8506a87"
format_stamp: "Formatted at 2026-09-29 16:09:17 on dist-test-slave-x07d"
server_key: "50317bc7df855036b91e3aba42a2f2d8"
server_key_iv: "09fd15227ac6edbd1337c9e9e6ed8bbd"
server_key_version: "encryptionkey@0"
I20260929 16:09:17.420831 5805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "4f411e2e371c4fb9b7655e8ec8506a87"
format_stamp: "Formatted at 2026-09-29 16:09:17 on dist-test-slave-x07d"
server_key: "50317bc7df855036b91e3aba42a2f2d8"
server_key_iv: "09fd15227ac6edbd1337c9e9e6ed8bbd"
server_key_version: "encryptionkey@0"
I20260929 16:09:17.427668 5805 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260929 16:09:17.432066 5820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:17.433641 5805 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260929 16:09:17.433897 5805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "4f411e2e371c4fb9b7655e8ec8506a87"
format_stamp: "Formatted at 2026-09-29 16:09:17 on dist-test-slave-x07d"
server_key: "50317bc7df855036b91e3aba42a2f2d8"
server_key_iv: "09fd15227ac6edbd1337c9e9e6ed8bbd"
server_key_version: "encryptionkey@0"
I20260929 16:09:17.434193 5805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:17.451220 5805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:17.456710 5805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:17.457096 5805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:17.458583 5805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:17.460237 5805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:17.460498 5805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:17.460719 5805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:17.460814 5805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:17.500519 5805 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:42485
I20260929 16:09:17.500594 5932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:42485 every 8 connection(s)
I20260929 16:09:17.502219 5805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:09:17.503460 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 5805
I20260929 16:09:17.503724 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:09:17.504689 20966 external_mini_cluster.cc:1428] Setting key 7a1b51edf5af7a1c933410906888d8f2
Sep 29 16:09:17 dist-test-slave-x07d krb5kdc[5699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:17.528424 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:17.505786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:44939 (local address 127.20.121.190:33233)
0929 16:09:17.506332 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:17.506337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:17.507938 (+ 1601us) server_negotiation.cc:407] Connection header received
0929 16:09:17.511207 (+ 3269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:17.511213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:17.511325 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:17.511441 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:17.513931 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:17.514981 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:17.517632 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:17.518133 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:17.518289 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:17.522234 (+ 3945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:17.522239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:17.522243 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:17.522280 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:17.525052 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:17.525878 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:17.525885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:17.525890 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:17.526081 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:17.526550 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:17.526556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:17.526559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:17.526856 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:17.527027 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:17.527800 (+ 773us) server_negotiation.cc:299] Negotiation successful
0929 16:09:17.528049 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":308,"threads_started":1}
I20260929 16:09:17.529470 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:17.506584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33233 (local address 127.20.121.129:44939)
0929 16:09:17.507707 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:17.507769 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:17.510757 (+ 2988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:17.511774 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:17.511789 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:17.512698 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:17.513646 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:17.513677 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:17.515203 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:17.515212 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:17.517428 (+ 2216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:17.517444 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:17.518414 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:17.518424 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:17.518776 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:17.519605 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:09:17.519629 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:17.521916 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:17.525225 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:17.525236 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:17.525256 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:17.525734 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:17.526197 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:17.526202 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:17.526206 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:17.526428 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:17.527003 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:17.527012 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:17.527405 (+ 393us) client_negotiation.cc:769] Sending connection context
0929 16:09:17.527691 (+ 286us) client_negotiation.cc:240] Negotiation successful
0929 16:09:17.528282 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":434,"threads_started":1}
I20260929 16:09:17.531216 5933 heartbeater.cc:347] Connected to a master server at 127.20.121.190:33233
I20260929 16:09:17.531539 5933 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
I20260929 16:09:17.532442 5933 heartbeater.cc:511] Master 127.20.121.190:33233 requested a full tablet report, sending...
I20260929 16:09:17.534829 5744 ts_manager.cc:194] Registered new tserver with Master: 4f411e2e371c4fb9b7655e8ec8506a87 (127.20.121.129:42485)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
I20260929 16:09:17.536784 5744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:44939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:09:17.569514 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33233
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:17.939896 5939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:17.940198 5939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:17.940402 5939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:17.948029 5939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:17.948163 5939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:17.948379 5939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:09:17.958619 5939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:17.960148 5939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:17.961565 5939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:17.975996 5947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:17.975931 5944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:17.975983 5945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:17.982285 5939 server_base.cc:1034] running on GCE node
I20260929 16:09:17.983012 5939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:17.984133 5939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:17.985352 5939 hybrid_clock.cc:648] HybridClock initialized: now 1790698157985279 us; error 59 us; skew 500 ppm
I20260929 16:09:17.985702 5939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:17 dist-test-slave-x07d krb5kdc[5699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:17.989143 5939 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:09:17.991031 5939 webserver.cc:460] Webserver started at http://127.20.121.130:40671/ using document root <none> and password file <none>
I20260929 16:09:17.991837 5939 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:17.991946 5939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:17.992259 5939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:17.995440 5939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "021210907d394ee6ad177d778ad13bac"
format_stamp: "Formatted at 2026-09-29 16:09:17 on dist-test-slave-x07d"
server_key: "e247eba4b341b43b870ccb30a272c3fc"
server_key_iv: "f021675a7739d6451cd09744ff03624b"
server_key_version: "encryptionkey@0"
I20260929 16:09:17.996289 5939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "021210907d394ee6ad177d778ad13bac"
format_stamp: "Formatted at 2026-09-29 16:09:17 on dist-test-slave-x07d"
server_key: "e247eba4b341b43b870ccb30a272c3fc"
server_key_iv: "f021675a7739d6451cd09744ff03624b"
server_key_version: "encryptionkey@0"
I20260929 16:09:18.003692 5939 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260929 16:09:18.008348 5954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:18.010182 5939 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260929 16:09:18.010411 5939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "021210907d394ee6ad177d778ad13bac"
format_stamp: "Formatted at 2026-09-29 16:09:17 on dist-test-slave-x07d"
server_key: "e247eba4b341b43b870ccb30a272c3fc"
server_key_iv: "f021675a7739d6451cd09744ff03624b"
server_key_version: "encryptionkey@0"
I20260929 16:09:18.010645 5939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:18.053484 5939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:18.058620 5939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:18.058988 5939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:18.060115 5939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:18.061611 5939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:18.061687 5939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:18.061828 5939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:18.061889 5939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:18.097342 5939 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:41055
I20260929 16:09:18.097393 6066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:41055 every 8 connection(s)
I20260929 16:09:18.098887 5939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:09:18.106736 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 5939
I20260929 16:09:18.106918 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:09:18.107379 20966 external_mini_cluster.cc:1428] Setting key c86dc18e996b9e11ad26e11a8858e9d6
Sep 29 16:09:18 dist-test-slave-x07d krb5kdc[5699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698157, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:18.123505 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.101633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:34893 (local address 127.20.121.190:33233)
0929 16:09:18.102121 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:18.102128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:18.103161 (+ 1033us) server_negotiation.cc:407] Connection header received
0929 16:09:18.105153 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:18.105157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:18.105254 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:18.105373 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:18.107443 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.108293 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.110650 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.111489 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.111649 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.115742 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:18.115751 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:18.115760 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:18.115812 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:18.119180 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:18.120379 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:18.120384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:18.120387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:18.120559 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:18.121169 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:18.121173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:18.121175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:18.121540 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:18.121669 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:18.122930 (+ 1261us) server_negotiation.cc:299] Negotiation successful
0929 16:09:18.123188 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":211,"threads_started":1}
I20260929 16:09:18.124873 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.102235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33233 (local address 127.20.121.130:34893)
0929 16:09:18.102972 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:18.103020 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:18.104796 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:18.105702 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:18.105712 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:18.106484 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:18.107233 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:18.107254 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.108553 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.108559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:18.110476 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:18.110493 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.111746 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.111752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:18.111990 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.112754 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:09:18.112789 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:18.115408 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:18.119382 (+ 3974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:18.119395 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:18.119420 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:18.120183 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:18.120696 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:18.120703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:18.120710 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:18.121000 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:18.121689 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:18.121701 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:18.122274 (+ 573us) client_negotiation.cc:769] Sending connection context
0929 16:09:18.122610 (+ 336us) client_negotiation.cc:240] Negotiation successful
0929 16:09:18.123281 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":277,"threads_started":1}
I20260929 16:09:18.126757 6067 heartbeater.cc:347] Connected to a master server at 127.20.121.190:33233
I20260929 16:09:18.127164 6067 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:18.127964 6067 heartbeater.cc:511] Master 127.20.121.190:33233 requested a full tablet report, sending...
I20260929 16:09:18.129703 5744 ts_manager.cc:194] Registered new tserver with Master: 021210907d394ee6ad177d778ad13bac (127.20.121.130:41055)
I20260929 16:09:18.131145 5744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:34893
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:09:18.178118 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33233
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:18.519486 6073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:18.519807 6073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:18.519974 6073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:18.527640 6073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:18.527765 6073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:18.528012 6073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:09:18.538363 6073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:18.539839 6073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:18.541251 5933 heartbeater.cc:503] Master 127.20.121.190:33233 was elected leader, sending a full tablet report...
I20260929 16:09:18.541366 6073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:18.555631 6078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:18.555644 6079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:18.555584 6081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:18.556831 6073 server_base.cc:1034] running on GCE node
I20260929 16:09:18.557569 6073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:18.558645 6073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:18.559866 6073 hybrid_clock.cc:648] HybridClock initialized: now 1790698158559789 us; error 56 us; skew 500 ppm
I20260929 16:09:18.560276 6073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:18 dist-test-slave-x07d krb5kdc[5699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698158, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:18.571619 6073 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:09:18.573498 6073 webserver.cc:460] Webserver started at http://127.20.121.131:36237/ using document root <none> and password file <none>
I20260929 16:09:18.574425 6073 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:18.574584 6073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:18.574947 6073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:18.578475 6073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "562ef58b49b14944adf3ed45178f8f82"
format_stamp: "Formatted at 2026-09-29 16:09:18 on dist-test-slave-x07d"
server_key: "3d085f1790ba9fadd7863d14b0bf4a45"
server_key_iv: "a49cc67fc263c1ec52f654a61b364d61"
server_key_version: "encryptionkey@0"
I20260929 16:09:18.579417 6073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "562ef58b49b14944adf3ed45178f8f82"
format_stamp: "Formatted at 2026-09-29 16:09:18 on dist-test-slave-x07d"
server_key: "3d085f1790ba9fadd7863d14b0bf4a45"
server_key_iv: "a49cc67fc263c1ec52f654a61b364d61"
server_key_version: "encryptionkey@0"
I20260929 16:09:18.585876 6073 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260929 16:09:18.590384 6088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:18.592298 6073 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 16:09:18.592520 6073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "562ef58b49b14944adf3ed45178f8f82"
format_stamp: "Formatted at 2026-09-29 16:09:18 on dist-test-slave-x07d"
server_key: "3d085f1790ba9fadd7863d14b0bf4a45"
server_key_iv: "a49cc67fc263c1ec52f654a61b364d61"
server_key_version: "encryptionkey@0"
I20260929 16:09:18.592784 6073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:18.622335 6073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:18.627466 6073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:18.627780 6073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:18.628808 6073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:18.630396 6073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:18.630475 6073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:18.630630 6073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:18.630682 6073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:18.666249 6073 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:36787
I20260929 16:09:18.666345 6200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:36787 every 8 connection(s)
I20260929 16:09:18.667614 6073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:09:18.674467 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 6073
I20260929 16:09:18.674677 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:09:18.675117 20966 external_mini_cluster.cc:1428] Setting key 1722753dba90b587fdac173e9a95606f
Sep 29 16:09:18 dist-test-slave-x07d krb5kdc[5699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698158, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:18.690977 6203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.670512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:45039 (local address 127.20.121.190:33233)
0929 16:09:18.671134 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:18.671163 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:18.671954 (+ 791us) server_negotiation.cc:407] Connection header received
0929 16:09:18.674055 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:18.674063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:18.674188 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:18.674334 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:18.676403 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.677626 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.679528 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.680015 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.680170 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.684351 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:18.684356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:18.684360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:18.684402 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:18.687342 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:18.688225 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:18.688234 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:18.688238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:18.688394 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:18.688880 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:18.688887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:18.688891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:18.689223 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:18.689437 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:18.690277 (+ 840us) server_negotiation.cc:299] Negotiation successful
0929 16:09:18.690538 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":376,"threads_started":1}
I20260929 16:09:18.692072 6204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.670896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33233 (local address 127.20.121.131:45039)
0929 16:09:18.671771 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:18.671822 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:18.673692 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:18.674640 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:18.674654 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:18.675396 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:18.676204 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:18.676226 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.677812 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.677823 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:18.679346 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:18.679361 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.680303 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.680313 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:18.680575 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.681459 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:09:18.681488 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:18.684066 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:18.687538 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:18.687552 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:18.687575 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:18.688075 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:18.688523 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:18.688529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:18.688534 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:18.688758 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:18.689418 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:18.689427 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:18.689818 (+ 391us) client_negotiation.cc:769] Sending connection context
0929 16:09:18.690186 (+ 368us) client_negotiation.cc:240] Negotiation successful
0929 16:09:18.690802 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":313,"threads_started":1}
I20260929 16:09:18.693558 6201 heartbeater.cc:347] Connected to a master server at 127.20.121.190:33233
I20260929 16:09:18.693991 6201 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:18.694888 6201 heartbeater.cc:511] Master 127.20.121.190:33233 requested a full tablet report, sending...
I20260929 16:09:18.696605 5744 ts_manager.cc:194] Registered new tserver with Master: 562ef58b49b14944adf3ed45178f8f82 (127.20.121.131:36787)
I20260929 16:09:18.697585 5744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:45039
I20260929 16:09:18.703504 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:09:18.722707 6203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.708432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.20.121.190:33233)
0929 16:09:18.708689 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:18.708696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:18.708849 (+ 153us) server_negotiation.cc:407] Connection header received
0929 16:09:18.709013 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:18.709017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:18.709098 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:18.709193 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:18.711068 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.711827 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.713227 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.713654 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.713760 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.716702 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:18.716708 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:18.716712 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:18.716755 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:18.719591 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:18.720456 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:18.720462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:18.720465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:18.720613 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:18.721120 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:18.721123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:18.721126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:18.721502 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:18.721720 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:18.722152 (+ 432us) server_negotiation.cc:299] Negotiation successful
0929 16:09:18.722344 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260929 16:09:18.727393 5744 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:09:18.734637 5744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:09:18.762395 6214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.749213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.20.121.129:42485)
0929 16:09:18.750362 (+ 1149us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:18.750375 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:18.751114 (+ 739us) server_negotiation.cc:407] Connection header received
0929 16:09:18.751610 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:18.751617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:18.751919 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:18.752151 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:18.753373 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.755218 (+ 1845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.758656 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.760536 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.760777 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.760939 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:18.761764 (+ 825us) server_negotiation.cc:299] Negotiation successful
0929 16:09:18.761988 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":857,"thread_start_us":1204,"threads_started":1}
I20260929 16:09:18.762434 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.749186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:42485 (local address 127.0.0.1:59168)
0929 16:09:18.750890 (+ 1704us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:18.750942 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:18.751319 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:18.752532 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:18.752541 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:18.752603 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:18.753149 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:18.753172 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.755375 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.755381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:18.758458 (+ 3077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:18.758472 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.761099 (+ 2627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.761106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:18.761438 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.761516 (+ 78us) client_negotiation.cc:769] Sending connection context
0929 16:09:18.761672 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 16:09:18.761912 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1307,"mutex_wait_us":52,"thread_start_us":2664,"threads_started":1}
I20260929 16:09:18.762442 6217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.750468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50698 (local address 127.20.121.130:41055)
0929 16:09:18.751216 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:18.751224 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:18.752573 (+ 1349us) server_negotiation.cc:407] Connection header received
0929 16:09:18.752694 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:18.752702 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:18.753044 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:18.753291 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:18.754456 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.756234 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.758925 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.760529 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.760730 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.760939 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:18.761860 (+ 921us) server_negotiation.cc:299] Negotiation successful
0929 16:09:18.762080 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":410,"threads_started":1}
I20260929 16:09:18.763255 6215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.749769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:41055 (local address 127.0.0.1:50698)
0929 16:09:18.752205 (+ 2436us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:18.752229 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:18.752379 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:18.753563 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:18.753571 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:18.753610 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:18.754113 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:18.754143 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.756414 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.756419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:18.758709 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:18.758729 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.761314 (+ 2585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.761324 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:18.761564 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.761591 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 16:09:18.761666 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 16:09:18.762781 (+ 1115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2292,"mutex_wait_us":900,"thread_start_us":913,"threads_started":1}
I20260929 16:09:18.764005 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.753412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:36787 (local address 127.0.0.1:42360)
0929 16:09:18.754666 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:18.754686 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:18.754821 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:18.755511 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:18.755517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:18.755559 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:18.756052 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:18.756065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.758336 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.758342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:18.760482 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:18.760499 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.763115 (+ 2616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.763122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:18.763456 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.763495 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 16:09:18.763592 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 16:09:18.763731 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1119,"thread_start_us":978,"threads_started":1}
I20260929 16:09:18.764972 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:18.752655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.20.121.131:36787)
0929 16:09:18.754101 (+ 1446us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:18.754113 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:18.754825 (+ 712us) server_negotiation.cc:407] Connection header received
0929 16:09:18.754952 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:18.754961 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:18.755287 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:18.755527 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:18.756309 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.758167 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.760699 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:18.762366 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:18.763002 (+ 636us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:18.763222 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:18.764353 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0929 16:09:18.764581 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1200,"thread_start_us":1010,"threads_started":1}
I20260929 16:09:18.767621 5868 tablet_service.cc:1467] Processing CreateTablet for tablet ea862e913c3846fea0171ba9aacbf64f (DEFAULT_TABLE table=test-table [id=b58e18fb259847348cee1e1f3f4b3e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:18.767678 6002 tablet_service.cc:1467] Processing CreateTablet for tablet ea862e913c3846fea0171ba9aacbf64f (DEFAULT_TABLE table=test-table [id=b58e18fb259847348cee1e1f3f4b3e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:18.768402 6136 tablet_service.cc:1467] Processing CreateTablet for tablet ea862e913c3846fea0171ba9aacbf64f (DEFAULT_TABLE table=test-table [id=b58e18fb259847348cee1e1f3f4b3e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:18.769342 6002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea862e913c3846fea0171ba9aacbf64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:18.769341 5868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea862e913c3846fea0171ba9aacbf64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:18.769778 6136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea862e913c3846fea0171ba9aacbf64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:18.785758 6220 tablet_bootstrap.cc:492] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: Bootstrap starting.
I20260929 16:09:18.787248 6222 tablet_bootstrap.cc:492] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: Bootstrap starting.
I20260929 16:09:18.787846 6221 tablet_bootstrap.cc:492] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: Bootstrap starting.
I20260929 16:09:18.790333 6220 tablet_bootstrap.cc:654] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:18.790963 6222 tablet_bootstrap.cc:654] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:18.791630 6221 tablet_bootstrap.cc:654] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:18.791879 6220 log.cc:824] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:18.792172 6222 log.cc:824] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:18.793515 6221 log.cc:824] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:18.795845 6222 tablet_bootstrap.cc:492] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: No bootstrap required, opened a new log
I20260929 16:09:18.795843 6220 tablet_bootstrap.cc:492] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: No bootstrap required, opened a new log
I20260929 16:09:18.796197 6220 ts_tablet_manager.cc:1397] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260929 16:09:18.796203 6222 ts_tablet_manager.cc:1397] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260929 16:09:18.798569 6221 tablet_bootstrap.cc:492] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: No bootstrap required, opened a new log
I20260929 16:09:18.798879 6221 ts_tablet_manager.cc:1397] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260929 16:09:18.808333 6221 raft_consensus.cc:348] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 } } peers { permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 } } peers { permanent_uuid: "562ef58b49b14944adf3ed45178f8f82" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36787 } }
I20260929 16:09:18.809105 6221 raft_consensus.cc:374] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:18.808979 6222 raft_consensus.cc:348] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 } } peers { permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 } } peers { permanent_uuid: "562ef58b49b14944adf3ed45178f8f82" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36787 } }
I20260929 16:09:18.809288 6221 raft_consensus.cc:729] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f411e2e371c4fb9b7655e8ec8506a87, State: Initialized, Role: FOLLOWER
I20260929 16:09:18.809491 6222 raft_consensus.cc:374] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:18.809613 6222 raft_consensus.cc:729] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021210907d394ee6ad177d778ad13bac, State: Initialized, Role: FOLLOWER
I20260929 16:09:18.809724 6220 raft_consensus.cc:348] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 } } peers { permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 } } peers { permanent_uuid: "562ef58b49b14944adf3ed45178f8f82" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36787 } }
I20260929 16:09:18.810267 6220 raft_consensus.cc:374] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:18.810472 6220 raft_consensus.cc:729] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 562ef58b49b14944adf3ed45178f8f82, State: Initialized, Role: FOLLOWER
I20260929 16:09:18.810506 6222 consensus_queue.cc:260] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 } } peers { permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 } } peers { permanent_uuid: "562ef58b49b14944adf3ed45178f8f82" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36787 } }
I20260929 16:09:18.810547 6221 consensus_queue.cc:260] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 } } peers { permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 } } peers { permanent_uuid: "562ef58b49b14944adf3ed45178f8f82" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36787 } }
I20260929 16:09:18.811275 6220 consensus_queue.cc:260] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 } } peers { permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 } } peers { permanent_uuid: "562ef58b49b14944adf3ed45178f8f82" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36787 } }
I20260929 16:09:18.819257 6222 ts_tablet_manager.cc:1428] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: Time spent starting tablet: real 0.023s user 0.017s sys 0.004s
I20260929 16:09:18.820369 6067 heartbeater.cc:503] Master 127.20.121.190:33233 was elected leader, sending a full tablet report...
I20260929 16:09:18.825116 6201 heartbeater.cc:503] Master 127.20.121.190:33233 was elected leader, sending a full tablet report...
I20260929 16:09:18.825982 6220 ts_tablet_manager.cc:1428] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: Time spent starting tablet: real 0.030s user 0.012s sys 0.014s
I20260929 16:09:18.827109 6221 ts_tablet_manager.cc:1428] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: Time spent starting tablet: real 0.028s user 0.017s sys 0.004s
W20260929 16:09:18.851100 6068 tablet.cc:2369] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:09:18.919409 6202 tablet.cc:2369] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:09:19.005918 5934 tablet.cc:2369] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:09:19.013196 6228 raft_consensus.cc:482] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:19.013523 6228 raft_consensus.cc:504] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 } } peers { permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 } } peers { permanent_uuid: "562ef58b49b14944adf3ed45178f8f82" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36787 } }
I20260929 16:09:19.015518 6228 leader_election.cc:288] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f411e2e371c4fb9b7655e8ec8506a87 (127.20.121.129:42485), 021210907d394ee6ad177d778ad13bac (127.20.121.130:41055)
I20260929 16:09:19.024047 6214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:19.016045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:47109 (local address 127.20.121.129:42485)
0929 16:09:19.016255 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:19.016261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:19.017339 (+ 1078us) server_negotiation.cc:407] Connection header received
0929 16:09:19.017509 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:19.017515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:19.017620 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:19.017722 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:19.018544 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.019616 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.021730 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.022834 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.023079 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:19.023194 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:19.023510 (+ 316us) server_negotiation.cc:299] Negotiation successful
0929 16:09:19.023703 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 16:09:19.024127 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:19.016198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:42485 (local address 127.20.121.131:47109)
0929 16:09:19.017205 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:19.017227 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:19.017367 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:19.017861 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:19.017870 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:19.017902 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:19.018352 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:19.018366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.019905 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.019910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:19.021552 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:19.021565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.023076 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.023084 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:19.023317 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:19.023358 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 16:09:19.023481 (+ 123us) client_negotiation.cc:240] Negotiation successful
0929 16:09:19.023642 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"mutex_wait_us":43,"thread_start_us":528,"threads_started":1}
I20260929 16:09:19.025125 6217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:19.016021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:46615 (local address 127.20.121.130:41055)
0929 16:09:19.016252 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:19.016259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:19.016613 (+ 354us) server_negotiation.cc:407] Connection header received
0929 16:09:19.016927 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:19.016934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:19.017027 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:19.017141 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:19.017817 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.018958 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.022200 (+ 3242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.023867 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.024031 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:19.024138 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:19.024687 (+ 549us) server_negotiation.cc:299] Negotiation successful
0929 16:09:19.024852 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 16:09:19.025113 6204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:19.015984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:41055 (local address 127.20.121.131:46615)
0929 16:09:19.016435 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:19.016462 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:19.016656 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:19.017190 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:19.017195 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:19.017221 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:19.017520 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:19.017533 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.019103 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.019109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:19.021826 (+ 2717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:19.021842 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.024084 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.024095 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:19.024472 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:19.024508 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 16:09:19.024607 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 16:09:19.024777 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":1}
I20260929 16:09:19.025759 5888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea862e913c3846fea0171ba9aacbf64f" candidate_uuid: "562ef58b49b14944adf3ed45178f8f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" is_pre_election: true
I20260929 16:09:19.026465 5888 raft_consensus.cc:2454] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 562ef58b49b14944adf3ed45178f8f82 in term 0.
I20260929 16:09:19.026355 6022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea862e913c3846fea0171ba9aacbf64f" candidate_uuid: "562ef58b49b14944adf3ed45178f8f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021210907d394ee6ad177d778ad13bac" is_pre_election: true
I20260929 16:09:19.026894 6022 raft_consensus.cc:2454] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 562ef58b49b14944adf3ed45178f8f82 in term 0.
I20260929 16:09:19.027230 6092 leader_election.cc:302] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f411e2e371c4fb9b7655e8ec8506a87, 562ef58b49b14944adf3ed45178f8f82; no voters:
I20260929 16:09:19.027741 6228 raft_consensus.cc:2781] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:09:19.027915 6228 raft_consensus.cc:482] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:09:19.028036 6228 raft_consensus.cc:3037] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:19.030526 6228 raft_consensus.cc:504] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 } } peers { permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 } } peers { permanent_uuid: "562ef58b49b14944adf3ed45178f8f82" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36787 } }
I20260929 16:09:19.031220 6228 leader_election.cc:288] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [CANDIDATE]: Term 1 election: Requested vote from peers 4f411e2e371c4fb9b7655e8ec8506a87 (127.20.121.129:42485), 021210907d394ee6ad177d778ad13bac (127.20.121.130:41055)
I20260929 16:09:19.031862 5888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea862e913c3846fea0171ba9aacbf64f" candidate_uuid: "562ef58b49b14944adf3ed45178f8f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f411e2e371c4fb9b7655e8ec8506a87"
I20260929 16:09:19.031867 6022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea862e913c3846fea0171ba9aacbf64f" candidate_uuid: "562ef58b49b14944adf3ed45178f8f82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021210907d394ee6ad177d778ad13bac"
I20260929 16:09:19.032125 6022 raft_consensus.cc:3037] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:19.032125 5888 raft_consensus.cc:3037] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:19.035883 6022 raft_consensus.cc:2454] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 562ef58b49b14944adf3ed45178f8f82 in term 1.
I20260929 16:09:19.035883 5888 raft_consensus.cc:2454] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 562ef58b49b14944adf3ed45178f8f82 in term 1.
I20260929 16:09:19.036495 6089 leader_election.cc:302] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021210907d394ee6ad177d778ad13bac, 562ef58b49b14944adf3ed45178f8f82; no voters:
I20260929 16:09:19.036895 6228 raft_consensus.cc:2781] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:19.037549 6228 raft_consensus.cc:686] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 1 LEADER]: Becoming Leader. State: Replica: 562ef58b49b14944adf3ed45178f8f82, State: Running, Role: LEADER
I20260929 16:09:19.038193 6228 consensus_queue.cc:237] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 } } peers { permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 } } peers { permanent_uuid: "562ef58b49b14944adf3ed45178f8f82" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36787 } }
I20260929 16:09:19.043454 5744 catalog_manager.cc:5495] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 562ef58b49b14944adf3ed45178f8f82 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "562ef58b49b14944adf3ed45178f8f82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "562ef58b49b14944adf3ed45178f8f82" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36787 } health_report { overall_health: HEALTHY } } }
I20260929 16:09:19.066246 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:19.060118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42364 (local address 127.20.121.131:36787)
0929 16:09:19.060326 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:19.060332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:19.060464 (+ 132us) server_negotiation.cc:407] Connection header received
0929 16:09:19.060718 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:19.060722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:19.060802 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:19.061065 (+ 263us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:09:19.061434 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.062228 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.063784 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.064318 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.064478 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:19.065030 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:09:19.065596 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:09:19.065753 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:19.065853 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 16:09:19.065945 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 16:09:19.075910 5888 raft_consensus.cc:1261] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87 [term 1 FOLLOWER]: Refusing update from remote peer 562ef58b49b14944adf3ed45178f8f82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:19.075910 6022 raft_consensus.cc:1261] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac [term 1 FOLLOWER]: Refusing update from remote peer 562ef58b49b14944adf3ed45178f8f82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:19.077106 6230 consensus_queue.cc:1035] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f411e2e371c4fb9b7655e8ec8506a87" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 42485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:19.077513 6228 consensus_queue.cc:1035] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "021210907d394ee6ad177d778ad13bac" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:19.116855 5744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:19.117203 5744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:19.120970 5744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3261065b215b4005936c920d24031f95: Sending DeleteTablet for 3 replicas of tablet ea862e913c3846fea0171ba9aacbf64f
I20260929 16:09:19.122257 5868 tablet_service.cc:1514] Processing DeleteTablet for tablet ea862e913c3846fea0171ba9aacbf64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:19 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:59168
I20260929 16:09:19.123046 6002 tablet_service.cc:1514] Processing DeleteTablet for tablet ea862e913c3846fea0171ba9aacbf64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:19 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:50698
I20260929 16:09:19.123203 6243 tablet_replica.cc:330] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: stopping tablet replica
I20260929 16:09:19.123826 6243 raft_consensus.cc:2229] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:19.124398 6136 tablet_service.cc:1514] Processing DeleteTablet for tablet ea862e913c3846fea0171ba9aacbf64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:19 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:42360
I20260929 16:09:19.124473 6243 raft_consensus.cc:2258] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:19.125954 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 5805
I20260929 16:09:19.127491 6243 ts_tablet_manager.cc:1905] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:19.137668 6243 ts_tablet_manager.cc:1918] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:19.138118 6243 log.cc:1196] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/ea862e913c3846fea0171ba9aacbf64f
I20260929 16:09:19.139050 6243 ts_tablet_manager.cc:1939] T ea862e913c3846fea0171ba9aacbf64f P 4f411e2e371c4fb9b7655e8ec8506a87: Deleting consensus metadata
I20260929 16:09:19.141412 5731 catalog_manager.cc:4841] TS 4f411e2e371c4fb9b7655e8ec8506a87 (127.20.121.129:42485): tablet ea862e913c3846fea0171ba9aacbf64f (table test-table [id=b58e18fb259847348cee1e1f3f4b3e51]) successfully deleted
I20260929 16:09:19.138772 6244 tablet_replica.cc:330] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: stopping tablet replica
I20260929 16:09:19.142284 6244 raft_consensus.cc:2229] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:19.142861 6244 raft_consensus.cc:2258] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:19.143535 6245 tablet_replica.cc:330] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: stopping tablet replica
I20260929 16:09:19.144008 6245 raft_consensus.cc:2229] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:09:19.144874 6245 raft_consensus.cc:2258] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:19.145371 6244 ts_tablet_manager.cc:1905] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:19.147832 6245 ts_tablet_manager.cc:1905] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 16:09:19 dist-test-slave-x07d krb5kdc[5699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:09:19.153105 6244 ts_tablet_manager.cc:1918] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:19.153319 6244 log.cc:1196] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/ea862e913c3846fea0171ba9aacbf64f
I20260929 16:09:19.153954 6244 ts_tablet_manager.cc:1939] T ea862e913c3846fea0171ba9aacbf64f P 021210907d394ee6ad177d778ad13bac: Deleting consensus metadata
I20260929 16:09:19.156203 5728 catalog_manager.cc:4841] TS 021210907d394ee6ad177d778ad13bac (127.20.121.130:41055): tablet ea862e913c3846fea0171ba9aacbf64f (table test-table [id=b58e18fb259847348cee1e1f3f4b3e51]) successfully deleted
I20260929 16:09:19.156574 6245 ts_tablet_manager.cc:1918] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:19.156769 6245 log.cc:1196] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/ea862e913c3846fea0171ba9aacbf64f
I20260929 16:09:19.157384 6245 ts_tablet_manager.cc:1939] T ea862e913c3846fea0171ba9aacbf64f P 562ef58b49b14944adf3ed45178f8f82: Deleting consensus metadata
I20260929 16:09:19.159390 5731 catalog_manager.cc:4841] TS 562ef58b49b14944adf3ed45178f8f82 (127.20.121.131:36787): tablet ea862e913c3846fea0171ba9aacbf64f (table test-table [id=b58e18fb259847348cee1e1f3f4b3e51]) successfully deleted
I20260929 16:09:19.160684 6214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:19.127473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59182 (local address 127.20.121.129:42485)
0929 16:09:19.127657 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:19.127663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:19.133540 (+ 5877us) server_negotiation.cc:407] Connection header received
0929 16:09:19.134581 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:19.134589 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:19.134687 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:19.134774 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:19.139062 (+ 4288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.140256 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.145867 (+ 5611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.146573 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.146688 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:19.153036 (+ 6348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:19.153060 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:19.153083 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:19.153144 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:19.157315 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:19.158248 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:19.158257 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:19.158259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:19.158417 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:19.158855 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:19.158867 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:19.158871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:19.159344 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:19.159776 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:19.159984 (+ 208us) server_negotiation.cc:299] Negotiation successful
0929 16:09:19.160333 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 16:09:19.161502 5928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:19.256652 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 5805
I20260929 16:09:19.281221 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 5939
Sep 29 16:09:19 dist-test-slave-x07d krb5kdc[5699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:09:19.303735 6217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:19.285010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50714 (local address 127.20.121.130:41055)
0929 16:09:19.285293 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:19.285302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:19.285638 (+ 336us) server_negotiation.cc:407] Connection header received
0929 16:09:19.285720 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:19.285726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:19.285845 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:19.285966 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:19.287857 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.289059 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.291038 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.291660 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.291786 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:19.297017 (+ 5231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:19.297034 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:19.297051 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:19.297101 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:19.300471 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:19.301329 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:19.301340 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:19.301344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:19.301495 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:19.301907 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:19.301912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:19.301916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:19.302420 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:19.302763 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:19.303111 (+ 348us) server_negotiation.cc:299] Negotiation successful
0929 16:09:19.303348 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260929 16:09:19.304477 6062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:19.403008 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 5939
I20260929 16:09:19.426721 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 6073
Sep 29 16:09:19 dist-test-slave-x07d krb5kdc[5699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:09:19.447376 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:19.427666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.20.121.131:36787)
0929 16:09:19.427839 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:19.427844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:19.427917 (+ 73us) server_negotiation.cc:407] Connection header received
0929 16:09:19.428181 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:19.428186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:19.428273 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:19.428403 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:19.430462 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.431436 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.433566 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:19.434409 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:19.434552 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:19.440526 (+ 5974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:19.440542 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:19.440557 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:19.440607 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:19.444140 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:19.444974 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:19.444983 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:19.444985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:19.445148 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:19.445564 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:19.445575 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:19.445582 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:19.445983 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:19.446397 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:19.446685 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 16:09:19.446955 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 16:09:19.448176 6196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:19.575513 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 6073
I20260929 16:09:19.592403 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 5712
I20260929 16:09:19.593811 5774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:19.711112 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 5712
2026-09-29T16:09:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4770 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T16:09:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:19Z Disabled control of system clock
I20260929 16:09:19.794723 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38635
--webserver_interface=127.20.121.190
--webserver_port=0
--builtin_ntp_servers=127.20.121.148:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:38635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 16:09:20.153491 6254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:20.153810 6254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:20.153895 6254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:20.161710 6254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:20.161844 6254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:20.161883 6254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:20.161934 6254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:20.161976 6254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:20.172014 6254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:38635
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:20.173336 6254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:20.175262 6254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:20.189102 6260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:20.189077 6259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:20.189209 6262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:20.190225 6254 server_base.cc:1034] running on GCE node
I20260929 16:09:20.191246 6254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:20.192917 6254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:20.194165 6254 hybrid_clock.cc:648] HybridClock initialized: now 1790698160194106 us; error 45 us; skew 500 ppm
I20260929 16:09:20.194635 6254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:20.197497 6254 webserver.cc:460] Webserver started at http://127.20.121.190:35845/ using document root <none> and password file <none>
I20260929 16:09:20.198450 6254 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:20.198611 6254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:20.198994 6254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:20.202678 6254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "96b39cab0d964094a60d613b3b4d0a3d"
format_stamp: "Formatted at 2026-09-29 16:09:20 on dist-test-slave-x07d"
server_key: "094b63f829379fc1847508d3db798315"
server_key_iv: "4a96408b17b1b12617a21f37fd9a7dbf"
server_key_version: "encryptionkey@0"
I20260929 16:09:20.203724 6254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "96b39cab0d964094a60d613b3b4d0a3d"
format_stamp: "Formatted at 2026-09-29 16:09:20 on dist-test-slave-x07d"
server_key: "094b63f829379fc1847508d3db798315"
server_key_iv: "4a96408b17b1b12617a21f37fd9a7dbf"
server_key_version: "encryptionkey@0"
I20260929 16:09:20.210099 6254 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 16:09:20.214332 6268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:20.215986 6254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:09:20.216217 6254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "96b39cab0d964094a60d613b3b4d0a3d"
format_stamp: "Formatted at 2026-09-29 16:09:20 on dist-test-slave-x07d"
server_key: "094b63f829379fc1847508d3db798315"
server_key_iv: "4a96408b17b1b12617a21f37fd9a7dbf"
server_key_version: "encryptionkey@0"
I20260929 16:09:20.216471 6254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:20.235317 6254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:20.236299 6254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:20.236634 6254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:20.257449 6254 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:38635
I20260929 16:09:20.257545 6319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:38635 every 8 connection(s)
I20260929 16:09:20.258916 6254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:20.261152 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 6254
I20260929 16:09:20.261404 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:20.262054 20966 external_mini_cluster.cc:1428] Setting key 236149d2031db5ebae5f22f9f153a93f
I20260929 16:09:20.264482 6320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:20.276971 6323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:20.264156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58530 (local address 127.20.121.190:38635)
0929 16:09:20.265344 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:20.265357 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:20.265413 (+ 56us) server_negotiation.cc:407] Connection header received
0929 16:09:20.267237 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:20.267263 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:20.267835 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:20.268303 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:20.268911 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:20.270423 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:20.271892 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:20.272658 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:20.272883 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:20.274394 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:20.274405 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:20.274423 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:20.275073 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:20.275175 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:20.275199 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:20.275348 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:20.275575 (+ 227us) server_negotiation.cc:299] Negotiation successful
0929 16:09:20.275938 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":478,"threads_started":1}
I20260929 16:09:20.277556 6320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d: Bootstrap starting.
I20260929 16:09:20.281131 6320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:20.282519 6320 log.cc:824] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:20.286001 6320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d: No bootstrap required, opened a new log
I20260929 16:09:20.293874 6320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b39cab0d964094a60d613b3b4d0a3d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38635 } }
I20260929 16:09:20.294340 6320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:20.294422 6320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b39cab0d964094a60d613b3b4d0a3d, State: Initialized, Role: FOLLOWER
I20260929 16:09:20.295028 6320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b39cab0d964094a60d613b3b4d0a3d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38635 } }
I20260929 16:09:20.295256 6320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:20.295358 6320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:20.295487 6320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:20.298313 6320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b39cab0d964094a60d613b3b4d0a3d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38635 } }
I20260929 16:09:20.298849 6320 leader_election.cc:302] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96b39cab0d964094a60d613b3b4d0a3d; no voters:
I20260929 16:09:20.299463 6320 leader_election.cc:288] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:20.299631 6325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:20.300359 6325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [term 1 LEADER]: Becoming Leader. State: Replica: 96b39cab0d964094a60d613b3b4d0a3d, State: Running, Role: LEADER
I20260929 16:09:20.300958 6325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b39cab0d964094a60d613b3b4d0a3d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38635 } }
I20260929 16:09:20.301730 6320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:20.303974 6327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96b39cab0d964094a60d613b3b4d0a3d. Latest consensus state: current_term: 1 leader_uuid: "96b39cab0d964094a60d613b3b4d0a3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b39cab0d964094a60d613b3b4d0a3d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38635 } } }
I20260929 16:09:20.304463 6327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:20.304505 6326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96b39cab0d964094a60d613b3b4d0a3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b39cab0d964094a60d613b3b4d0a3d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38635 } } }
I20260929 16:09:20.304816 6326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:20.305639 6332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:20.311748 6332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:20.322119 6332 catalog_manager.cc:1347] Generated new cluster ID: fc15cc83630949d3beaeeb07caa88a37
I20260929 16:09:20.322253 6332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:20.336151 6332 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:20.338109 6332 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:20.348340 6332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d: Generated new TSK 0
I20260929 16:09:20.349535 6332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:20.356848 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38635
--builtin_ntp_servers=127.20.121.148:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 16:09:20.743793 6344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:20.744156 6344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:20.744329 6344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:20.752014 6344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:20.752182 6344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:20.752362 6344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:09:20.763329 6344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:20.764899 6344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:20.767189 6344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:20.784091 6350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:20.784341 6349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:20.784370 6352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:20.785965 6344 server_base.cc:1034] running on GCE node
I20260929 16:09:20.786746 6344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:20.787911 6344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:20.789171 6344 hybrid_clock.cc:648] HybridClock initialized: now 1790698160789113 us; error 56 us; skew 500 ppm
I20260929 16:09:20.789533 6344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:20.792390 6344 webserver.cc:460] Webserver started at http://127.20.121.129:38087/ using document root <none> and password file <none>
I20260929 16:09:20.793186 6344 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:20.793330 6344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:20.793669 6344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:20.797258 6344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "352f66cc40ff4c33befeaed35e9860ff"
format_stamp: "Formatted at 2026-09-29 16:09:20 on dist-test-slave-x07d"
server_key: "53d9f2f72809358836b93e7186206c8c"
server_key_iv: "57aa9e156e4d02ed95ca64cccac5e948"
server_key_version: "encryptionkey@0"
I20260929 16:09:20.798375 6344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "352f66cc40ff4c33befeaed35e9860ff"
format_stamp: "Formatted at 2026-09-29 16:09:20 on dist-test-slave-x07d"
server_key: "53d9f2f72809358836b93e7186206c8c"
server_key_iv: "57aa9e156e4d02ed95ca64cccac5e948"
server_key_version: "encryptionkey@0"
I20260929 16:09:20.804585 6344 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260929 16:09:20.808806 6358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:20.810554 6344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 16:09:20.810772 6344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "352f66cc40ff4c33befeaed35e9860ff"
format_stamp: "Formatted at 2026-09-29 16:09:20 on dist-test-slave-x07d"
server_key: "53d9f2f72809358836b93e7186206c8c"
server_key_iv: "57aa9e156e4d02ed95ca64cccac5e948"
server_key_version: "encryptionkey@0"
I20260929 16:09:20.811007 6344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:20.828243 6344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:20.829090 6344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:20.829365 6344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:20.830456 6344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:20.831995 6344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:20.832088 6344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:20.832165 6344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:20.832219 6344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:20.867934 6344 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:45903
I20260929 16:09:20.868031 6470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:45903 every 8 connection(s)
I20260929 16:09:20.869335 6344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:09:20.872725 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 6344
I20260929 16:09:20.872999 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:09:20.873660 20966 external_mini_cluster.cc:1428] Setting key 79f3d8dd02231fa21c93145bac0a46a6
I20260929 16:09:20.877084 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38635
--builtin_ntp_servers=127.20.121.148:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 16:09:20.885857 6473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:20.872181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:44399 (local address 127.20.121.190:38635)
0929 16:09:20.872733 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:20.872740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:20.874329 (+ 1589us) server_negotiation.cc:407] Connection header received
0929 16:09:20.876764 (+ 2435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:20.876768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:20.876837 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:20.876934 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:20.878711 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:20.879477 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:20.882403 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:20.882908 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:20.883020 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:20.885025 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:20.885030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:20.885033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:20.885055 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:20.885101 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:20.885113 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:20.885183 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:20.885499 (+ 316us) server_negotiation.cc:299] Negotiation successful
0929 16:09:20.885592 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":259,"threads_started":1}
I20260929 16:09:20.887303 6474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:20.872723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38635 (local address 127.20.121.129:44399)
0929 16:09:20.874039 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:20.874112 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:20.876403 (+ 2291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:20.877271 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:20.877299 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:20.877447 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:20.878478 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:20.878511 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:20.879669 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:20.879679 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:20.882178 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:20.882200 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:20.883203 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:20.883213 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:20.883614 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:20.884525 (+ 911us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:09:20.884532 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:20.884600 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:09:20.884724 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:20.885221 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:20.885236 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:20.885249 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 16:09:20.885468 (+ 219us) client_negotiation.cc:240] Negotiation successful
0929 16:09:20.885996 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":431,"threads_started":1}
I20260929 16:09:20.889164 6471 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38635
I20260929 16:09:20.889554 6471 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:20.890457 6471 heartbeater.cc:511] Master 127.20.121.190:38635 requested a full tablet report, sending...
I20260929 16:09:20.893018 6285 ts_manager.cc:194] Registered new tserver with Master: 352f66cc40ff4c33befeaed35e9860ff (127.20.121.129:45903)
I20260929 16:09:20.894770 6285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.129:44399
W20260929 16:09:21.240789 6475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:21.241175 6475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:21.241359 6475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:21.249545 6475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:21.249722 6475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:21.249912 6475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:09:21.259487 6475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:21.260782 6475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:21.262677 6475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:21.277215 6481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:21.277275 6483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:21.277555 6475 server_base.cc:1034] running on GCE node
W20260929 16:09:21.277184 6480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:21.278554 6475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:21.279601 6475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:21.280870 6475 hybrid_clock.cc:648] HybridClock initialized: now 1790698161280817 us; error 51 us; skew 500 ppm
I20260929 16:09:21.281213 6475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:21.283806 6475 webserver.cc:460] Webserver started at http://127.20.121.130:39841/ using document root <none> and password file <none>
I20260929 16:09:21.284556 6475 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:21.284673 6475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:21.285014 6475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:21.288669 6475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "e4b7d3d95ad649029b4c5e582ffe4d40"
format_stamp: "Formatted at 2026-09-29 16:09:21 on dist-test-slave-x07d"
server_key: "414e12376505d7deb376027ce28ba4a2"
server_key_iv: "fd0ea68eac5f05091244c85d61101f95"
server_key_version: "encryptionkey@0"
I20260929 16:09:21.289495 6475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "e4b7d3d95ad649029b4c5e582ffe4d40"
format_stamp: "Formatted at 2026-09-29 16:09:21 on dist-test-slave-x07d"
server_key: "414e12376505d7deb376027ce28ba4a2"
server_key_iv: "fd0ea68eac5f05091244c85d61101f95"
server_key_version: "encryptionkey@0"
I20260929 16:09:21.294987 6475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 16:09:21.299122 6489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:21.300530 6475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:09:21.300740 6475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "e4b7d3d95ad649029b4c5e582ffe4d40"
format_stamp: "Formatted at 2026-09-29 16:09:21 on dist-test-slave-x07d"
server_key: "414e12376505d7deb376027ce28ba4a2"
server_key_iv: "fd0ea68eac5f05091244c85d61101f95"
server_key_version: "encryptionkey@0"
I20260929 16:09:21.300997 6475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:21.324617 6475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:21.325587 6475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:21.325968 6475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:21.327194 6475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:21.328876 6475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:21.329012 6475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:21.329159 6475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:21.329272 6475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:21.366590 6475 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:35809
I20260929 16:09:21.366711 6601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:35809 every 8 connection(s)
I20260929 16:09:21.368182 6475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:09:21.377769 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 6475
I20260929 16:09:21.378000 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:09:21.378587 20966 external_mini_cluster.cc:1428] Setting key 6b64381d4f2ffdf4995c2856c8a18e88
I20260929 16:09:21.381902 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38635
--builtin_ntp_servers=127.20.121.148:41749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 16:09:21.383832 6473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.371319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:50313 (local address 127.20.121.190:38635)
0929 16:09:21.371563 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:21.371570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:21.373227 (+ 1657us) server_negotiation.cc:407] Connection header received
0929 16:09:21.375352 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:21.375359 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:21.375438 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:21.375541 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:21.376738 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.377805 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.380137 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.380794 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.380937 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.382769 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:21.382779 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:21.382788 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:21.382817 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:21.382869 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:21.382890 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:21.383001 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:21.383244 (+ 243us) server_negotiation.cc:299] Negotiation successful
0929 16:09:21.383375 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 16:09:21.384888 6604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.372037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38635 (local address 127.20.121.130:50313)
0929 16:09:21.373041 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:21.373087 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:21.375037 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:21.375768 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:21.375778 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:21.375900 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:21.376554 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:21.376573 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.378003 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.378041 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:21.379895 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:21.379910 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.381056 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.381062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:21.381304 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.382379 (+ 1075us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:09:21.382382 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:21.382430 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:09:21.382525 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:21.383002 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:21.383010 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:21.383017 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 16:09:21.383167 (+ 150us) client_negotiation.cc:240] Negotiation successful
0929 16:09:21.383545 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":311,"threads_started":1}
I20260929 16:09:21.386634 6602 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38635
I20260929 16:09:21.387022 6602 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:21.388026 6602 heartbeater.cc:511] Master 127.20.121.190:38635 requested a full tablet report, sending...
I20260929 16:09:21.389708 6285 ts_manager.cc:194] Registered new tserver with Master: e4b7d3d95ad649029b4c5e582ffe4d40 (127.20.121.130:35809)
I20260929 16:09:21.390638 6285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.130:50313
W20260929 16:09:21.735107 6605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:21.735450 6605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:21.735548 6605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:21.744062 6605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:21.744203 6605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:21.744324 6605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:09:21.754405 6605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:41749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38635
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:21.755826 6605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:21.757812 6605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:21.772197 6610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:21.772292 6611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:21.772248 6613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:21.773391 6605 server_base.cc:1034] running on GCE node
I20260929 16:09:21.774176 6605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:21.775185 6605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:21.776433 6605 hybrid_clock.cc:648] HybridClock initialized: now 1790698161776390 us; error 40 us; skew 500 ppm
I20260929 16:09:21.776808 6605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:21.779455 6605 webserver.cc:460] Webserver started at http://127.20.121.131:39729/ using document root <none> and password file <none>
I20260929 16:09:21.780236 6605 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:21.780329 6605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:21.780689 6605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:21.783861 6605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "669fd7cb2d7647099e684d6c68b54318"
format_stamp: "Formatted at 2026-09-29 16:09:21 on dist-test-slave-x07d"
server_key: "e52fad0b84c92c60c46a01cd052c0889"
server_key_iv: "b61ca5a4d0dad0ec066011339ac97358"
server_key_version: "encryptionkey@0"
I20260929 16:09:21.784720 6605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "669fd7cb2d7647099e684d6c68b54318"
format_stamp: "Formatted at 2026-09-29 16:09:21 on dist-test-slave-x07d"
server_key: "e52fad0b84c92c60c46a01cd052c0889"
server_key_iv: "b61ca5a4d0dad0ec066011339ac97358"
server_key_version: "encryptionkey@0"
I20260929 16:09:21.790329 6605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 16:09:21.793928 6619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:21.795629 6605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:09:21.795908 6605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "669fd7cb2d7647099e684d6c68b54318"
format_stamp: "Formatted at 2026-09-29 16:09:21 on dist-test-slave-x07d"
server_key: "e52fad0b84c92c60c46a01cd052c0889"
server_key_iv: "b61ca5a4d0dad0ec066011339ac97358"
server_key_version: "encryptionkey@0"
I20260929 16:09:21.796154 6605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:21.813887 6605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:21.814893 6605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:21.815228 6605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:21.816308 6605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:21.817906 6605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:21.818053 6605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:21.818198 6605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:21.818290 6605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:21.854715 6605 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:41695
I20260929 16:09:21.854830 6731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:41695 every 8 connection(s)
I20260929 16:09:21.856178 6605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:09:21.861218 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 6605
I20260929 16:09:21.861447 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:09:21.862192 20966 external_mini_cluster.cc:1428] Setting key cf058721aee3064aee402be72f0622a3
I20260929 16:09:21.871416 6473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.859248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:35265 (local address 127.20.121.190:38635)
0929 16:09:21.859462 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:21.859468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:21.861081 (+ 1613us) server_negotiation.cc:407] Connection header received
0929 16:09:21.863900 (+ 2819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:21.863906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:21.863978 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:21.864115 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:21.865560 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.866381 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.868080 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.868730 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.868854 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.870468 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:21.870475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:21.870480 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:21.870505 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:21.870545 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:21.870562 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:21.870665 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:21.870974 (+ 309us) server_negotiation.cc:299] Negotiation successful
0929 16:09:21.871082 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 16:09:21.872484 6734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.859820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38635 (local address 127.20.121.131:35265)
0929 16:09:21.860863 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:21.860929 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:21.863500 (+ 2571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:21.864457 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:21.864473 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:21.864577 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:21.865382 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:21.865411 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.866519 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.866524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:21.867929 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:21.867941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.868962 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.868968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:21.869184 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.870138 (+ 954us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:09:21.870141 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:21.870181 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:09:21.870267 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:21.870691 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:21.870700 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:21.870707 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 16:09:21.870934 (+ 227us) client_negotiation.cc:240] Negotiation successful
0929 16:09:21.871306 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":301,"threads_started":1}
I20260929 16:09:21.873980 6732 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38635
I20260929 16:09:21.874351 6732 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:21.875154 6732 heartbeater.cc:511] Master 127.20.121.190:38635 requested a full tablet report, sending...
I20260929 16:09:21.877084 6285 ts_manager.cc:194] Registered new tserver with Master: 669fd7cb2d7647099e684d6c68b54318 (127.20.121.131:41695)
I20260929 16:09:21.877964 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:09:21.878363 6285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.131:35265
I20260929 16:09:21.890137 6473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.883454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.20.121.190:38635)
0929 16:09:21.883698 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:21.883704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:21.883963 (+ 259us) server_negotiation.cc:407] Connection header received
0929 16:09:21.884119 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:21.884123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:21.884189 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:21.884275 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:21.884795 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.885506 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.887358 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.887870 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.888021 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.889228 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:21.889233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:21.889237 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:21.889257 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:21.889297 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:21.889309 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:21.889408 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:21.889578 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 16:09:21.889741 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 16:09:21.898358 6471 heartbeater.cc:503] Master 127.20.121.190:38635 was elected leader, sending a full tablet report...
I20260929 16:09:21.894722 6285 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:09:21.902487 6285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:09:21.945904 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.917577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.20.121.131:41695)
0929 16:09:21.936256 (+ 18679us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:21.936269 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:21.936318 (+ 49us) server_negotiation.cc:407] Connection header received
0929 16:09:21.936576 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:21.936590 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:21.937141 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:21.938458 (+ 1317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:21.938821 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.940663 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.942886 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.944292 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.944519 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.944754 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:21.945293 (+ 539us) server_negotiation.cc:299] Negotiation successful
0929 16:09:21.945484 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18443,"thread_start_us":18249,"threads_started":1}
I20260929 16:09:21.945945 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.917241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:41695 (local address 127.0.0.1:36376)
0929 16:09:21.924305 (+ 7064us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:21.924365 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:21.924550 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:21.937502 (+ 12952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:21.937514 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:21.937562 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:21.938176 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:21.938195 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.940854 (+ 2659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.940859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:21.942687 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:21.942701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.944557 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.944564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:21.944968 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.945047 (+ 79us) client_negotiation.cc:769] Sending connection context
0929 16:09:21.945196 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 16:09:21.945431 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":61,"thread_start_us":519,"threads_started":1}
I20260929 16:09:21.948644 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.917869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:35809 (local address 127.0.0.1:59868)
0929 16:09:21.933368 (+ 15499us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:21.933404 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:21.935427 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:21.940552 (+ 5125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:21.940564 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:21.940591 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:21.941177 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:21.941190 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.943439 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.943445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:21.945676 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:21.945692 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.947659 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.947668 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:21.947847 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.947892 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 16:09:21.947978 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 16:09:21.948118 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15277,"mutex_wait_us":46,"thread_start_us":17223,"threads_started":1}
I20260929 16:09:21.949545 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.918512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59868 (local address 127.20.121.130:35809)
0929 16:09:21.939902 (+ 21390us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:21.939911 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:21.939948 (+ 37us) server_negotiation.cc:407] Connection header received
0929 16:09:21.940053 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:21.940061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:21.940289 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:21.941432 (+ 1143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:21.941736 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.943262 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.945944 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.947160 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.947354 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.948939 (+ 1585us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:21.949026 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 16:09:21.949196 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21210,"thread_start_us":21074,"threads_started":1}
I20260929 16:09:21.950066 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.919614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.20.121.129:45903)
0929 16:09:21.938626 (+ 19012us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:21.938642 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:21.939300 (+ 658us) server_negotiation.cc:407] Connection header received
0929 16:09:21.939514 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:21.939528 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:21.939813 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:21.940041 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:21.941302 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.943055 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.946547 (+ 3492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.948155 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.948386 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.948627 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:21.949465 (+ 838us) server_negotiation.cc:299] Negotiation successful
0929 16:09:21.949646 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18772,"thread_start_us":18611,"threads_started":1}
I20260929 16:09:21.950403 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:21.918938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45903 (local address 127.0.0.1:46326)
0929 16:09:21.939143 (+ 20205us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:21.939166 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:21.939336 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:21.940355 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:21.940361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:21.940395 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:21.940963 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:21.940989 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.943256 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.943261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:21.945736 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:21.945758 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:21.949048 (+ 3290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:21.949058 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:21.949249 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:21.949276 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 16:09:21.949403 (+ 127us) client_negotiation.cc:240] Negotiation successful
0929 16:09:21.949547 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20063,"mutex_wait_us":260,"thread_start_us":19907,"threads_started":1}
I20260929 16:09:21.950969 6667 tablet_service.cc:1467] Processing CreateTablet for tablet 76aab9143a124e298b56bf9d14195dd3 (DEFAULT_TABLE table=test-table [id=4608dafb34bd4390a4021b94e7de5005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:21.952198 6667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76aab9143a124e298b56bf9d14195dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:21.953729 6537 tablet_service.cc:1467] Processing CreateTablet for tablet 76aab9143a124e298b56bf9d14195dd3 (DEFAULT_TABLE table=test-table [id=4608dafb34bd4390a4021b94e7de5005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:21.954187 6406 tablet_service.cc:1467] Processing CreateTablet for tablet 76aab9143a124e298b56bf9d14195dd3 (DEFAULT_TABLE table=test-table [id=4608dafb34bd4390a4021b94e7de5005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:21.955330 6537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76aab9143a124e298b56bf9d14195dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:21.955616 6406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76aab9143a124e298b56bf9d14195dd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:21.963984 6750 tablet_bootstrap.cc:492] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: Bootstrap starting.
I20260929 16:09:21.967607 6750 tablet_bootstrap.cc:654] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:21.969086 6750 log.cc:824] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:21.971982 6752 tablet_bootstrap.cc:492] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: Bootstrap starting.
I20260929 16:09:21.975266 6753 tablet_bootstrap.cc:492] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: Bootstrap starting.
I20260929 16:09:21.977237 6752 tablet_bootstrap.cc:654] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:21.978833 6753 tablet_bootstrap.cc:654] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:21.979339 6752 log.cc:824] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:21.980280 6753 log.cc:824] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:21.982550 6750 tablet_bootstrap.cc:492] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: No bootstrap required, opened a new log
I20260929 16:09:21.982736 6752 tablet_bootstrap.cc:492] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: No bootstrap required, opened a new log
I20260929 16:09:21.982965 6750 ts_tablet_manager.cc:1397] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.007s
I20260929 16:09:21.983018 6752 ts_tablet_manager.cc:1397] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260929 16:09:21.984004 6753 tablet_bootstrap.cc:492] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: No bootstrap required, opened a new log
I20260929 16:09:21.984290 6753 ts_tablet_manager.cc:1397] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 16:09:21.991361 6750 raft_consensus.cc:348] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669fd7cb2d7647099e684d6c68b54318" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 41695 } } peers { permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 } } peers { permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 } }
I20260929 16:09:21.991882 6750 raft_consensus.cc:374] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:21.992000 6750 raft_consensus.cc:729] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 669fd7cb2d7647099e684d6c68b54318, State: Initialized, Role: FOLLOWER
I20260929 16:09:21.992816 6750 consensus_queue.cc:260] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669fd7cb2d7647099e684d6c68b54318" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 41695 } } peers { permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 } } peers { permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 } }
I20260929 16:09:21.994239 6732 heartbeater.cc:503] Master 127.20.121.190:38635 was elected leader, sending a full tablet report...
I20260929 16:09:21.995007 6750 ts_tablet_manager.cc:1428] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260929 16:09:21.995139 6752 raft_consensus.cc:348] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669fd7cb2d7647099e684d6c68b54318" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 41695 } } peers { permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 } } peers { permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 } }
I20260929 16:09:21.995676 6752 raft_consensus.cc:374] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:21.995822 6752 raft_consensus.cc:729] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 352f66cc40ff4c33befeaed35e9860ff, State: Initialized, Role: FOLLOWER
I20260929 16:09:21.995792 6753 raft_consensus.cc:348] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669fd7cb2d7647099e684d6c68b54318" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 41695 } } peers { permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 } } peers { permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 } }
I20260929 16:09:21.996291 6753 raft_consensus.cc:374] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:21.996399 6753 raft_consensus.cc:729] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4b7d3d95ad649029b4c5e582ffe4d40, State: Initialized, Role: FOLLOWER
I20260929 16:09:21.996570 6752 consensus_queue.cc:260] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669fd7cb2d7647099e684d6c68b54318" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 41695 } } peers { permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 } } peers { permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 } }
I20260929 16:09:21.997035 6753 consensus_queue.cc:260] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669fd7cb2d7647099e684d6c68b54318" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 41695 } } peers { permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 } } peers { permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 } }
I20260929 16:09:21.999528 6602 heartbeater.cc:503] Master 127.20.121.190:38635 was elected leader, sending a full tablet report...
I20260929 16:09:22.001368 6752 ts_tablet_manager.cc:1428] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
I20260929 16:09:21.999619 6753 ts_tablet_manager.cc:1428] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260929 16:09:22.108258 6733 tablet.cc:2369] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:09:22.120636 6603 tablet.cc:2369] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:09:22.121945 6472 tablet.cc:2369] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:09:22.305413 6756 raft_consensus.cc:482] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:22.305756 6756 raft_consensus.cc:504] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669fd7cb2d7647099e684d6c68b54318" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 41695 } } peers { permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 } } peers { permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 } }
I20260929 16:09:22.307829 6756 leader_election.cc:288] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 352f66cc40ff4c33befeaed35e9860ff (127.20.121.129:45903), e4b7d3d95ad649029b4c5e582ffe4d40 (127.20.121.130:35809)
I20260929 16:09:22.314685 6734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:22.308102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:35809 (local address 127.20.121.131:49117)
0929 16:09:22.308431 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:22.308451 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:22.308605 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:22.309004 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:22.309009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:22.309035 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:22.309278 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:22.309291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.310820 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.310825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:22.312435 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:22.312448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.313777 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.313783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:22.313979 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:22.314037 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 16:09:22.314165 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 16:09:22.314390 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186}
I20260929 16:09:22.315085 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:22.308257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:49117 (local address 127.20.121.130:35809)
0929 16:09:22.308437 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:22.308445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:22.308561 (+ 116us) server_negotiation.cc:407] Connection header received
0929 16:09:22.308777 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:22.308783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:22.308861 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:22.308981 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:22.309630 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.310675 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.312627 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.313593 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.313722 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:22.313776 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:22.314230 (+ 454us) server_negotiation.cc:299] Negotiation successful
0929 16:09:22.314447 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 16:09:22.315385 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:22.308289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:45903 (local address 127.20.121.131:33815)
0929 16:09:22.309054 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:22.309070 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:22.309189 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:22.309667 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:22.309672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:22.309705 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:22.310102 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:22.310125 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.311385 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.311389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:22.312933 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:22.312947 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.314440 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.314446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:22.314645 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:22.314682 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 16:09:22.314912 (+ 230us) client_negotiation.cc:240] Negotiation successful
0929 16:09:22.315058 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":430,"threads_started":1}
I20260929 16:09:22.315393 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:22.308355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:33815 (local address 127.20.121.129:45903)
0929 16:09:22.308628 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:22.308635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:22.309186 (+ 551us) server_negotiation.cc:407] Connection header received
0929 16:09:22.309311 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:22.309319 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:22.309514 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:22.309617 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:22.310331 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.311201 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.313170 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.314273 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.314419 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:22.314495 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:22.314963 (+ 468us) server_negotiation.cc:299] Negotiation successful
0929 16:09:22.315127 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260929 16:09:22.316095 6557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76aab9143a124e298b56bf9d14195dd3" candidate_uuid: "669fd7cb2d7647099e684d6c68b54318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" is_pre_election: true
I20260929 16:09:22.316404 6426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76aab9143a124e298b56bf9d14195dd3" candidate_uuid: "669fd7cb2d7647099e684d6c68b54318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "352f66cc40ff4c33befeaed35e9860ff" is_pre_election: true
I20260929 16:09:22.316780 6557 raft_consensus.cc:2454] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 669fd7cb2d7647099e684d6c68b54318 in term 0.
I20260929 16:09:22.316960 6426 raft_consensus.cc:2454] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 669fd7cb2d7647099e684d6c68b54318 in term 0.
I20260929 16:09:22.317600 6621 leader_election.cc:302] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 669fd7cb2d7647099e684d6c68b54318, e4b7d3d95ad649029b4c5e582ffe4d40; no voters:
I20260929 16:09:22.318163 6756 raft_consensus.cc:2781] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:09:22.318395 6756 raft_consensus.cc:482] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:09:22.318528 6756 raft_consensus.cc:3037] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:22.321230 6756 raft_consensus.cc:504] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669fd7cb2d7647099e684d6c68b54318" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 41695 } } peers { permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 } } peers { permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 } }
I20260929 16:09:22.321981 6756 leader_election.cc:288] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [CANDIDATE]: Term 1 election: Requested vote from peers 352f66cc40ff4c33befeaed35e9860ff (127.20.121.129:45903), e4b7d3d95ad649029b4c5e582ffe4d40 (127.20.121.130:35809)
I20260929 16:09:22.322650 6426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76aab9143a124e298b56bf9d14195dd3" candidate_uuid: "669fd7cb2d7647099e684d6c68b54318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "352f66cc40ff4c33befeaed35e9860ff"
I20260929 16:09:22.322650 6557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76aab9143a124e298b56bf9d14195dd3" candidate_uuid: "669fd7cb2d7647099e684d6c68b54318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40"
I20260929 16:09:22.322899 6426 raft_consensus.cc:3037] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:22.322886 6557 raft_consensus.cc:3037] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:22.326318 6426 raft_consensus.cc:2454] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 669fd7cb2d7647099e684d6c68b54318 in term 1.
I20260929 16:09:22.326318 6557 raft_consensus.cc:2454] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 669fd7cb2d7647099e684d6c68b54318 in term 1.
I20260929 16:09:22.326995 6621 leader_election.cc:302] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 352f66cc40ff4c33befeaed35e9860ff, 669fd7cb2d7647099e684d6c68b54318, e4b7d3d95ad649029b4c5e582ffe4d40; no voters:
I20260929 16:09:22.327479 6756 raft_consensus.cc:2781] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:22.328084 6756 raft_consensus.cc:686] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 1 LEADER]: Becoming Leader. State: Replica: 669fd7cb2d7647099e684d6c68b54318, State: Running, Role: LEADER
I20260929 16:09:22.328706 6756 consensus_queue.cc:237] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669fd7cb2d7647099e684d6c68b54318" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 41695 } } peers { permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 } } peers { permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 } }
I20260929 16:09:22.333957 6282 catalog_manager.cc:5495] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 reported cstate change: term changed from 0 to 1, leader changed from <none> to 669fd7cb2d7647099e684d6c68b54318 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "669fd7cb2d7647099e684d6c68b54318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669fd7cb2d7647099e684d6c68b54318" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 41695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 } health_report { overall_health: UNKNOWN } } }
I20260929 16:09:22.396019 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:22.388968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.20.121.131:41695)
0929 16:09:22.389253 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:22.389258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:22.389279 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:09:22.389383 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:22.389388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:22.389475 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:22.389633 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:09:22.390436 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.391240 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.393113 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.393589 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.393724 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:22.394709 (+ 985us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:09:22.395388 (+ 679us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:09:22.395591 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:22.395650 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 16:09:22.395742 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260929 16:09:22.406162 6557 raft_consensus.cc:1261] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40 [term 1 FOLLOWER]: Refusing update from remote peer 669fd7cb2d7647099e684d6c68b54318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:22.406842 6426 raft_consensus.cc:1261] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff [term 1 FOLLOWER]: Refusing update from remote peer 669fd7cb2d7647099e684d6c68b54318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:22.407428 6760 consensus_queue.cc:1035] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4b7d3d95ad649029b4c5e582ffe4d40" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 35809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:22.407989 6756 consensus_queue.cc:1035] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "352f66cc40ff4c33befeaed35e9860ff" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 45903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:22.442765 6282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:22.443075 6282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:22.447324 6282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96b39cab0d964094a60d613b3b4d0a3d: Sending DeleteTablet for 3 replicas of tablet 76aab9143a124e298b56bf9d14195dd3
I20260929 16:09:22.448844 6537 tablet_service.cc:1514] Processing DeleteTablet for tablet 76aab9143a124e298b56bf9d14195dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:22 UTC) from {username='slave'} at 127.0.0.1:59868
I20260929 16:09:22.448844 6406 tablet_service.cc:1514] Processing DeleteTablet for tablet 76aab9143a124e298b56bf9d14195dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:22 UTC) from {username='slave'} at 127.0.0.1:46326
I20260929 16:09:22.449105 6667 tablet_service.cc:1514] Processing DeleteTablet for tablet 76aab9143a124e298b56bf9d14195dd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:22 UTC) from {username='slave'} at 127.0.0.1:36376
I20260929 16:09:22.451586 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 6344
I20260929 16:09:22.465288 6775 tablet_replica.cc:330] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: stopping tablet replica
I20260929 16:09:22.465509 6774 tablet_replica.cc:330] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: stopping tablet replica
I20260929 16:09:22.466058 6774 raft_consensus.cc:2229] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:22.465984 6775 raft_consensus.cc:2229] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:09:22.466713 6774 raft_consensus.cc:2258] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:22.467005 6775 raft_consensus.cc:2258] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:22.469599 6774 ts_tablet_manager.cc:1905] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:22.469609 6775 ts_tablet_manager.cc:1905] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:22.477346 6773 tablet_replica.cc:330] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: stopping tablet replica
I20260929 16:09:22.477985 6775 ts_tablet_manager.cc:1918] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:22.478277 6775 log.cc:1196] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/76aab9143a124e298b56bf9d14195dd3
I20260929 16:09:22.478289 6774 ts_tablet_manager.cc:1918] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:22.478519 6774 log.cc:1196] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/76aab9143a124e298b56bf9d14195dd3
I20260929 16:09:22.478883 6775 ts_tablet_manager.cc:1939] T 76aab9143a124e298b56bf9d14195dd3 P 669fd7cb2d7647099e684d6c68b54318: Deleting consensus metadata
I20260929 16:09:22.479218 6774 ts_tablet_manager.cc:1939] T 76aab9143a124e298b56bf9d14195dd3 P 352f66cc40ff4c33befeaed35e9860ff: Deleting consensus metadata
I20260929 16:09:22.478083 6773 raft_consensus.cc:2229] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:22.480931 6271 catalog_manager.cc:4841] TS 669fd7cb2d7647099e684d6c68b54318 (127.20.121.131:41695): tablet 76aab9143a124e298b56bf9d14195dd3 (table test-table [id=4608dafb34bd4390a4021b94e7de5005]) successfully deleted
I20260929 16:09:22.481485 6773 raft_consensus.cc:2258] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:22.481546 6271 catalog_manager.cc:4841] TS 352f66cc40ff4c33befeaed35e9860ff (127.20.121.129:45903): tablet 76aab9143a124e298b56bf9d14195dd3 (table test-table [id=4608dafb34bd4390a4021b94e7de5005]) successfully deleted
I20260929 16:09:22.484194 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:22.453082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.20.121.129:45903)
0929 16:09:22.453318 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:22.453327 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:22.472155 (+ 18828us) server_negotiation.cc:407] Connection header received
0929 16:09:22.472706 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:22.472713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:22.472825 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:22.472909 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:22.473635 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.474818 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.478614 (+ 3796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.479375 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.479523 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:22.482291 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:22.482308 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:22.482327 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:22.483192 (+ 865us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:22.483262 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:22.483299 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:22.483499 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:22.483678 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 16:09:22.483813 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 16:09:22.484459 6773 ts_tablet_manager.cc:1905] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:22.485270 6466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:22.493623 6773 ts_tablet_manager.cc:1918] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:22.493868 6773 log.cc:1196] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/76aab9143a124e298b56bf9d14195dd3
I20260929 16:09:22.494536 6773 ts_tablet_manager.cc:1939] T 76aab9143a124e298b56bf9d14195dd3 P e4b7d3d95ad649029b4c5e582ffe4d40: Deleting consensus metadata
I20260929 16:09:22.496831 6270 catalog_manager.cc:4841] TS e4b7d3d95ad649029b4c5e582ffe4d40 (127.20.121.130:35809): tablet 76aab9143a124e298b56bf9d14195dd3 (table test-table [id=4608dafb34bd4390a4021b94e7de5005]) successfully deleted
I20260929 16:09:22.584097 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 6344
I20260929 16:09:22.602910 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 6475
I20260929 16:09:22.612215 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:22.603858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59872 (local address 127.20.121.130:35809)
0929 16:09:22.604056 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:22.604063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:22.604159 (+ 96us) server_negotiation.cc:407] Connection header received
0929 16:09:22.604366 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:22.604372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:22.604516 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:22.604642 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:22.605197 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.606423 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.607899 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.608565 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.608704 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:22.610301 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:22.610315 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:22.610336 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:22.611262 (+ 926us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:22.611331 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:22.611358 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:22.611504 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:22.611683 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 16:09:22.611831 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 16:09:22.613062 6597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:22.727141 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 6475
I20260929 16:09:22.750785 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 6605
I20260929 16:09:22.759806 6745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:22.751616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36398 (local address 127.20.121.131:41695)
0929 16:09:22.751845 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:22.751853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:22.751914 (+ 61us) server_negotiation.cc:407] Connection header received
0929 16:09:22.751991 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:22.751997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:22.752113 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:22.752273 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:22.752888 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.754224 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.755758 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:22.756488 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:22.756631 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:22.757936 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:22.757948 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:22.757968 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:22.758872 (+ 904us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:22.758945 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:22.758978 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:22.759104 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:22.759288 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 16:09:22.759394 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 16:09:22.760654 6727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:22.885236 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 6605
I20260929 16:09:22.906147 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 6254
I20260929 16:09:22.907478 6315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:23.029217 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 6254
2026-09-29T16:09:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3311 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:23 dist-test-slave-x07d krb5kdc[6783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:23 dist-test-slave-x07d krb5kdc[6783](info): set up 2 sockets
Sep 29 16:09:23 dist-test-slave-x07d krb5kdc[6783](info): commencing operation
krb5kdc: starting...
W20260929 16:09:24.178529 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.008s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:24 dist-test-slave-x07d krb5kdc[6783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:09:24.458771 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38163
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:38163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:24.811370 6796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:24.811679 6796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:24.811867 6796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:24.819398 6796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:24.819545 6796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:24.819641 6796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:24.819764 6796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:24.819847 6796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:24.830253 6796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:38163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:38163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:24.831691 6796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:24.833425 6796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:24.846774 6801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:24.846747 6804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:24.846686 6802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:24.847723 6796 server_base.cc:1034] running on GCE node
I20260929 16:09:24.848681 6796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:24.850627 6796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:24.851907 6796 hybrid_clock.cc:648] HybridClock initialized: now 1790698164851852 us; error 44 us; skew 500 ppm
I20260929 16:09:24.852384 6796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:24 dist-test-slave-x07d krb5kdc[6783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698164, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:24.863392 6796 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:09:24.865357 6796 webserver.cc:460] Webserver started at http://127.20.121.190:39733/ using document root <none> and password file <none>
I20260929 16:09:24.866210 6796 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:24.866374 6796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:24.866811 6796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:24.870108 6796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "53eac6c762c34cbf8deed65d5f7512c3"
format_stamp: "Formatted at 2026-09-29 16:09:24 on dist-test-slave-x07d"
server_key: "d47ba64ddc3c77862462d801e3efba26"
server_key_iv: "225b7ff8fb7b838e8cbb3589e2dd658e"
server_key_version: "encryptionkey@0"
I20260929 16:09:24.870935 6796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "53eac6c762c34cbf8deed65d5f7512c3"
format_stamp: "Formatted at 2026-09-29 16:09:24 on dist-test-slave-x07d"
server_key: "d47ba64ddc3c77862462d801e3efba26"
server_key_iv: "225b7ff8fb7b838e8cbb3589e2dd658e"
server_key_version: "encryptionkey@0"
I20260929 16:09:24.876421 6796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:09:24.880172 6811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:24.881857 6796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:09:24.882145 6796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "53eac6c762c34cbf8deed65d5f7512c3"
format_stamp: "Formatted at 2026-09-29 16:09:24 on dist-test-slave-x07d"
server_key: "d47ba64ddc3c77862462d801e3efba26"
server_key_iv: "225b7ff8fb7b838e8cbb3589e2dd658e"
server_key_version: "encryptionkey@0"
I20260929 16:09:24.882401 6796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:24.900204 6796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:24.904965 6796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:24.905299 6796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:24.925414 6796 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:38163
I20260929 16:09:24.925460 6862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:38163 every 8 connection(s)
I20260929 16:09:24.927026 6796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:24.931041 6863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:24.936949 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 6796
I20260929 16:09:24.937162 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:24.937673 20966 external_mini_cluster.cc:1428] Setting key fe518c67f6165dac0e48f22bc9c5900c
I20260929 16:09:24.943128 6863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3: Bootstrap starting.
I20260929 16:09:24.946481 6863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:24.947549 6863 log.cc:824] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:24.951145 6863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3: No bootstrap required, opened a new log
Sep 29 16:09:24 dist-test-slave-x07d krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:24.962210 6863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53eac6c762c34cbf8deed65d5f7512c3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38163 } }
I20260929 16:09:24.962754 6863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:24.962883 6863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53eac6c762c34cbf8deed65d5f7512c3, State: Initialized, Role: FOLLOWER
I20260929 16:09:24.963608 6863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53eac6c762c34cbf8deed65d5f7512c3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38163 } }
I20260929 16:09:24.963922 6863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:24.964071 6863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:24.964260 6863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:24.964728 6866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:24.939892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44384 (local address 127.20.121.190:38163)
0929 16:09:24.940855 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:24.940868 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:24.940923 (+ 55us) server_negotiation.cc:407] Connection header received
0929 16:09:24.942871 (+ 1948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:24.942896 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:24.943386 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:24.943817 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:24.945432 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:24.946944 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:24.948605 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:24.949606 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:24.949798 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:24.954523 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:24.954535 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:24.954561 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:24.954625 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:24.959548 (+ 4923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:24.960476 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:24.960486 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:24.960496 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:24.960747 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:24.961209 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:24.961215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:24.961218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:24.962047 (+ 829us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:24.962364 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:24.962768 (+ 404us) server_negotiation.cc:299] Negotiation successful
0929 16:09:24.963261 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":436,"threads_started":1}
I20260929 16:09:24.968097 6863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53eac6c762c34cbf8deed65d5f7512c3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38163 } }
I20260929 16:09:24.968659 6863 leader_election.cc:302] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53eac6c762c34cbf8deed65d5f7512c3; no voters:
I20260929 16:09:24.969296 6863 leader_election.cc:288] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:24.969450 6868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:24.970218 6868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [term 1 LEADER]: Becoming Leader. State: Replica: 53eac6c762c34cbf8deed65d5f7512c3, State: Running, Role: LEADER
I20260929 16:09:24.970904 6868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53eac6c762c34cbf8deed65d5f7512c3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38163 } }
I20260929 16:09:24.971503 6863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:24.973644 6870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53eac6c762c34cbf8deed65d5f7512c3. Latest consensus state: current_term: 1 leader_uuid: "53eac6c762c34cbf8deed65d5f7512c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53eac6c762c34cbf8deed65d5f7512c3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38163 } } }
I20260929 16:09:24.974123 6870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:24.975138 6869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53eac6c762c34cbf8deed65d5f7512c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53eac6c762c34cbf8deed65d5f7512c3" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 38163 } } }
I20260929 16:09:24.976588 6869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:24.977902 6873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:24.984844 6873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:24.995609 6873 catalog_manager.cc:1347] Generated new cluster ID: 3b741c5e2f7f449f8b3af02c560f8e91
I20260929 16:09:24.995747 6873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:25.031498 6873 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:25.033514 6873 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:25.050643 6873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3: Generated new TSK 0
I20260929 16:09:25.051779 6873 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:09:25.118220 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38163
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:25.469192 6889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:25.469602 6889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:25.469774 6889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:25.477885 6889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:25.478082 6889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:25.478289 6889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:09:25.489145 6889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:25.490712 6889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:25.492202 6889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:25.506767 6897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:25.506822 6895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:25.506867 6894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:25.507865 6889 server_base.cc:1034] running on GCE node
I20260929 16:09:25.508512 6889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:25.509575 6889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:25.510883 6889 hybrid_clock.cc:648] HybridClock initialized: now 1790698165510777 us; error 103 us; skew 500 ppm
I20260929 16:09:25.511273 6889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:25 dist-test-slave-x07d krb5kdc[6783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698165, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:25.514786 6889 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:09:25.516456 6889 webserver.cc:460] Webserver started at http://127.20.121.129:40135/ using document root <none> and password file <none>
I20260929 16:09:25.517146 6889 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:25.517251 6889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:25.517527 6889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:25.520632 6889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "0b2643340b254dcc9419de3d4d134706"
format_stamp: "Formatted at 2026-09-29 16:09:25 on dist-test-slave-x07d"
server_key: "9f1a701ad7603a5c8c4e67477eb28252"
server_key_iv: "ee6131f28c787fb6efebd7e90132ae97"
server_key_version: "encryptionkey@0"
I20260929 16:09:25.521407 6889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "0b2643340b254dcc9419de3d4d134706"
format_stamp: "Formatted at 2026-09-29 16:09:25 on dist-test-slave-x07d"
server_key: "9f1a701ad7603a5c8c4e67477eb28252"
server_key_iv: "ee6131f28c787fb6efebd7e90132ae97"
server_key_version: "encryptionkey@0"
I20260929 16:09:25.527287 6889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 16:09:25.531798 6904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:25.533638 6889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:09:25.533812 6889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "0b2643340b254dcc9419de3d4d134706"
format_stamp: "Formatted at 2026-09-29 16:09:25 on dist-test-slave-x07d"
server_key: "9f1a701ad7603a5c8c4e67477eb28252"
server_key_iv: "ee6131f28c787fb6efebd7e90132ae97"
server_key_version: "encryptionkey@0"
I20260929 16:09:25.534068 6889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:25.555665 6889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:25.560015 6889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:25.560283 6889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:25.561251 6889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:25.562601 6889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:25.562677 6889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:25.562762 6889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:25.562815 6889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:25.598239 6889 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:35071
I20260929 16:09:25.598297 7016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:35071 every 8 connection(s)
I20260929 16:09:25.599768 6889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:09:25.607626 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 6889
I20260929 16:09:25.607841 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:09:25.608323 20966 external_mini_cluster.cc:1428] Setting key b5305a30fd4a1076a6644d6d5498a878
Sep 29 16:09:25 dist-test-slave-x07d krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698165, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:25.621914 7019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:25.602554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:49547 (local address 127.20.121.190:38163)
0929 16:09:25.603041 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:25.603046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:25.603908 (+ 862us) server_negotiation.cc:407] Connection header received
0929 16:09:25.605829 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:25.605836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:25.605935 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:25.606083 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:25.608000 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:25.608850 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:25.610703 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:25.611129 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:25.611232 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:25.615744 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:25.615749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:25.615753 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:25.615789 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:25.618623 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:25.619471 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:25.619476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:25.619478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:25.619598 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:25.620058 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:25.620065 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:25.620068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:25.620355 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:25.620519 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:25.621266 (+ 747us) server_negotiation.cc:299] Negotiation successful
0929 16:09:25.621522 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":262,"threads_started":1}
I20260929 16:09:25.622997 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:25.602974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38163 (local address 127.20.121.129:49547)
0929 16:09:25.603671 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:25.603710 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:25.605526 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:25.606344 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:25.606353 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:25.606917 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:25.607701 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:25.607729 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:25.609043 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:25.609047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:25.610552 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:25.610568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:25.611379 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:25.611389 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:25.611757 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:25.612567 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:09:25.612602 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:25.615462 (+ 2860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:25.618799 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:25.618809 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:25.618829 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:25.619297 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:25.619718 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:25.619723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:25.619726 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:25.619940 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:25.620489 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:25.620497 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:25.620879 (+ 382us) client_negotiation.cc:769] Sending connection context
0929 16:09:25.621170 (+ 291us) client_negotiation.cc:240] Negotiation successful
0929 16:09:25.621771 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":265,"threads_started":1}
I20260929 16:09:25.624775 7017 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38163
I20260929 16:09:25.625108 7017 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:25.625871 7017 heartbeater.cc:511] Master 127.20.121.190:38163 requested a full tablet report, sending...
I20260929 16:09:25.628140 6828 ts_manager.cc:194] Registered new tserver with Master: 0b2643340b254dcc9419de3d4d134706 (127.20.121.129:35071)
I20260929 16:09:25.629626 6828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:49547
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:09:25.671025 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38163
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:26.021085 7023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:26.021438 7023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:26.021584 7023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:26.030040 7023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:26.030174 7023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:26.030289 7023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:09:26.041430 7023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:26.043336 7023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:26.044719 7023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:26.058642 7028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:26.058894 7031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:26.058929 7029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:26.059547 7023 server_base.cc:1034] running on GCE node
I20260929 16:09:26.060182 7023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:26.061234 7023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:26.062470 7023 hybrid_clock.cc:648] HybridClock initialized: now 1790698166062426 us; error 36 us; skew 500 ppm
I20260929 16:09:26.062790 7023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:26 dist-test-slave-x07d krb5kdc[6783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698166, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:26.066495 7023 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:09:26.068168 7023 webserver.cc:460] Webserver started at http://127.20.121.130:39787/ using document root <none> and password file <none>
I20260929 16:09:26.068943 7023 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:26.069083 7023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:26.069410 7023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:26.072561 7023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "7f392c62849f4da9ae6c5429ef149170"
format_stamp: "Formatted at 2026-09-29 16:09:26 on dist-test-slave-x07d"
server_key: "02c2625661cac4fd80f3e5ad83401d22"
server_key_iv: "7f91385202fe5e74bb2caf48e87e8f08"
server_key_version: "encryptionkey@0"
I20260929 16:09:26.073336 7023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "7f392c62849f4da9ae6c5429ef149170"
format_stamp: "Formatted at 2026-09-29 16:09:26 on dist-test-slave-x07d"
server_key: "02c2625661cac4fd80f3e5ad83401d22"
server_key_iv: "7f91385202fe5e74bb2caf48e87e8f08"
server_key_version: "encryptionkey@0"
I20260929 16:09:26.078926 7023 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 16:09:26.082926 7038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:26.084578 7023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:09:26.084798 7023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "7f392c62849f4da9ae6c5429ef149170"
format_stamp: "Formatted at 2026-09-29 16:09:26 on dist-test-slave-x07d"
server_key: "02c2625661cac4fd80f3e5ad83401d22"
server_key_iv: "7f91385202fe5e74bb2caf48e87e8f08"
server_key_version: "encryptionkey@0"
I20260929 16:09:26.085049 7023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:26.124135 7023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:26.129045 7023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:26.129374 7023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:26.130523 7023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:26.131994 7023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:26.132102 7023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:26.132256 7023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:26.132328 7023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:26.183284 7023 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:40271
I20260929 16:09:26.183388 7150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:40271 every 8 connection(s)
I20260929 16:09:26.184616 7023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:09:26.189795 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 7023
I20260929 16:09:26.190042 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:09:26.190614 20966 external_mini_cluster.cc:1428] Setting key 28e8487c4be0eed7aad9cf87a96a3708
Sep 29 16:09:26 dist-test-slave-x07d krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698166, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:26.208391 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.187508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:47075 (local address 127.20.121.190:38163)
0929 16:09:26.187982 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:26.187987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:26.188960 (+ 973us) server_negotiation.cc:407] Connection header received
0929 16:09:26.191118 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:26.191124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:26.191229 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:26.191352 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:26.193338 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.194145 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.195786 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.196242 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.196387 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.200288 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:26.200294 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:26.200297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:26.200336 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:26.204213 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:26.205151 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:26.205160 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:26.205166 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:26.205347 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:26.205975 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:26.205984 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:26.205990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:26.206440 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:26.206603 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:26.207566 (+ 963us) server_negotiation.cc:299] Negotiation successful
0929 16:09:26.207867 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":263,"threads_started":1}
I20260929 16:09:26.209194 7154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.188075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38163 (local address 127.20.121.130:47075)
0929 16:09:26.188781 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:26.188818 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:26.190817 (+ 1999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:26.191664 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:26.191677 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:26.192456 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:26.193123 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:26.193154 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.194304 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.194309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:26.195612 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:26.195626 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.196509 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.196517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:26.196768 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.197524 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:09:26.197548 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:26.199978 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:26.204441 (+ 4463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:26.204450 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:26.204468 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:26.204992 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:26.205526 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:26.205530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:26.205533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:26.205821 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:26.206610 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:26.206617 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:26.207058 (+ 441us) client_negotiation.cc:769] Sending connection context
0929 16:09:26.207426 (+ 368us) client_negotiation.cc:240] Negotiation successful
0929 16:09:26.207900 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":292,"threads_started":1}
I20260929 16:09:26.210989 7151 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38163
I20260929 16:09:26.211356 7151 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:26.212163 7151 heartbeater.cc:511] Master 127.20.121.190:38163 requested a full tablet report, sending...
I20260929 16:09:26.213963 6828 ts_manager.cc:194] Registered new tserver with Master: 7f392c62849f4da9ae6c5429ef149170 (127.20.121.130:40271)
I20260929 16:09:26.214946 6828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:47075
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:09:26.253705 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38163
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:40097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:26.597270 7157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:26.597573 7157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:26.597733 7157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:26.605635 7157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:26.605767 7157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:26.605945 7157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:09:26.617287 7157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:40097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:38163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:26.618906 7157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:26.620324 7157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 16:09:26.632975 7017 heartbeater.cc:503] Master 127.20.121.190:38163 was elected leader, sending a full tablet report...
W20260929 16:09:26.634745 7162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:26.634684 7163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:26.634663 7165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:26.635811 7157 server_base.cc:1034] running on GCE node
I20260929 16:09:26.636536 7157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:26.637754 7157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:26.639030 7157 hybrid_clock.cc:648] HybridClock initialized: now 1790698166638926 us; error 80 us; skew 500 ppm
I20260929 16:09:26.639452 7157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:26 dist-test-slave-x07d krb5kdc[6783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698166, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:26.643232 7157 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:09:26.645227 7157 webserver.cc:460] Webserver started at http://127.20.121.131:42873/ using document root <none> and password file <none>
I20260929 16:09:26.646126 7157 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:26.646311 7157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:26.646673 7157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:26.650344 7157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "d8e07570e3e14ce0931d26cdb1b80649"
format_stamp: "Formatted at 2026-09-29 16:09:26 on dist-test-slave-x07d"
server_key: "0441ca81c7a6cf8cc73d1829874d2cc8"
server_key_iv: "b23d4db83810596fd8c1965bff7aedc5"
server_key_version: "encryptionkey@0"
I20260929 16:09:26.651298 7157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "d8e07570e3e14ce0931d26cdb1b80649"
format_stamp: "Formatted at 2026-09-29 16:09:26 on dist-test-slave-x07d"
server_key: "0441ca81c7a6cf8cc73d1829874d2cc8"
server_key_iv: "b23d4db83810596fd8c1965bff7aedc5"
server_key_version: "encryptionkey@0"
I20260929 16:09:26.657462 7157 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260929 16:09:26.661988 7172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:26.663848 7157 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:09:26.664083 7157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "d8e07570e3e14ce0931d26cdb1b80649"
format_stamp: "Formatted at 2026-09-29 16:09:26 on dist-test-slave-x07d"
server_key: "0441ca81c7a6cf8cc73d1829874d2cc8"
server_key_iv: "b23d4db83810596fd8c1965bff7aedc5"
server_key_version: "encryptionkey@0"
I20260929 16:09:26.664331 7157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:26.679895 7157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:26.684638 7157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:26.685066 7157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:26.686220 7157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:26.687940 7157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:26.688043 7157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:26.688197 7157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:26.688279 7157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:26.722986 7157 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:45635
I20260929 16:09:26.723023 7284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:45635 every 8 connection(s)
I20260929 16:09:26.724406 7157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:09:26.731060 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 7157
I20260929 16:09:26.731271 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:09:26.731843 20966 external_mini_cluster.cc:1428] Setting key 2e6be0abed8ce5a6ed173203ad6706e2
Sep 29 16:09:26 dist-test-slave-x07d krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698166, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:26.748447 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.727692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:42831 (local address 127.20.121.190:38163)
0929 16:09:26.728253 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:26.728260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:26.729204 (+ 944us) server_negotiation.cc:407] Connection header received
0929 16:09:26.731469 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:26.731474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:26.731576 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:26.731687 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:26.733841 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.734878 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.736667 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.737088 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.737203 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.741423 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:26.741428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:26.741432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:26.741469 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:26.744207 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:26.745071 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:26.745075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:26.745077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:26.745208 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:26.745847 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:26.745852 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:26.745855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:26.746247 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:26.746408 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:26.747531 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0929 16:09:26.747902 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":275,"threads_started":1}
I20260929 16:09:26.749383 7288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.728198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:38163 (local address 127.20.121.131:42831)
0929 16:09:26.729011 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:26.729061 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:26.731049 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:26.732016 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:26.732028 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:26.732722 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:26.733547 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:26.733577 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.735077 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.735081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:26.736518 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:26.736531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.737284 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.737288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:26.737516 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.738267 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:09:26.738290 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:26.741137 (+ 2847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:26.744399 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:26.744408 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:26.744424 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:26.744923 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:26.745357 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:26.745361 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:26.745365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:26.745650 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:26.746416 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:26.746424 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:26.746970 (+ 546us) client_negotiation.cc:769] Sending connection context
0929 16:09:26.747349 (+ 379us) client_negotiation.cc:240] Negotiation successful
0929 16:09:26.747946 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":315,"threads_started":1}
I20260929 16:09:26.751077 7285 heartbeater.cc:347] Connected to a master server at 127.20.121.190:38163
I20260929 16:09:26.751502 7285 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:26.752368 7285 heartbeater.cc:511] Master 127.20.121.190:38163 requested a full tablet report, sending...
I20260929 16:09:26.754287 6828 ts_manager.cc:194] Registered new tserver with Master: d8e07570e3e14ce0931d26cdb1b80649 (127.20.121.131:45635)
I20260929 16:09:26.755175 6828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:42831
I20260929 16:09:26.761319 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:09:26.781181 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.765933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44386 (local address 127.20.121.190:38163)
0929 16:09:26.766228 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:26.766233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:26.766609 (+ 376us) server_negotiation.cc:407] Connection header received
0929 16:09:26.766736 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:26.766740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:26.766833 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:26.766965 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:26.768449 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.769264 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.770834 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.771275 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.771475 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.774723 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:26.774730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:26.774734 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:26.774776 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:26.777897 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:26.778956 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:26.778961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:26.778963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:26.779099 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:26.779661 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:26.779665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:26.779667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:26.779998 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:26.780299 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:26.780670 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 16:09:26.780898 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260929 16:09:26.785341 6828 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:09:26.792614 6828 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:09:26.818172 7300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.805814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35870 (local address 127.20.121.131:45635)
0929 16:09:26.807680 (+ 1866us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:26.807694 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:26.809973 (+ 2279us) server_negotiation.cc:407] Connection header received
0929 16:09:26.810221 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:26.810230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:26.810526 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:26.810712 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:26.811604 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.813249 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.815768 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.816924 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.817121 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.817316 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:26.817617 (+ 301us) server_negotiation.cc:299] Negotiation successful
0929 16:09:26.817794 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1623,"thread_start_us":1404,"threads_started":1}
I20260929 16:09:26.818225 7299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.805429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:45635 (local address 127.0.0.1:35870)
0929 16:09:26.809837 (+ 4408us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:26.809856 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:26.809980 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:26.810842 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:26.810846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:26.810866 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:26.811423 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:26.811433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.813416 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.813421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:26.815587 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:26.815602 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.817135 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.817141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:26.817342 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.817416 (+ 74us) client_negotiation.cc:769] Sending connection context
0929 16:09:26.817572 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 16:09:26.817789 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2610,"mutex_wait_us":194,"thread_start_us":3366,"threads_started":1}
I20260929 16:09:26.819563 7298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.805628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35071 (local address 127.0.0.1:52808)
0929 16:09:26.809154 (+ 3526us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:26.809192 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:26.809383 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:26.810790 (+ 1407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:26.810799 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:26.810841 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:26.811427 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:26.811440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.814060 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.814067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:26.816513 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:26.816529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.818702 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.818708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:26.818894 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.818926 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 16:09:26.819025 (+ 99us) client_negotiation.cc:240] Negotiation successful
0929 16:09:26.819163 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3343,"mutex_wait_us":54,"thread_start_us":2253,"threads_started":1}
I20260929 16:09:26.819607 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.805842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.20.121.129:35071)
0929 16:09:26.810075 (+ 4233us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:26.810087 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:26.810120 (+ 33us) server_negotiation.cc:407] Connection header received
0929 16:09:26.810226 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:26.810234 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:26.810534 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:26.811684 (+ 1150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:26.811963 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.813877 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.816707 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.818199 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.818410 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.818566 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:26.819098 (+ 532us) server_negotiation.cc:299] Negotiation successful
0929 16:09:26.819269 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3996,"thread_start_us":3811,"threads_started":1}
I20260929 16:09:26.821180 7302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.809195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:40271 (local address 127.0.0.1:53150)
0929 16:09:26.810893 (+ 1698us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:26.810910 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:26.811046 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:26.811722 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:26.811727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:26.811749 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:26.812086 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:26.812095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.813830 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.813836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:26.816513 (+ 2677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:26.816529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.820447 (+ 3918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.820451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:26.820567 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.820587 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 16:09:26.820697 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 16:09:26.820828 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1550,"mutex_wait_us":56,"thread_start_us":1392,"threads_started":1}
I20260929 16:09:26.821396 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:26.809755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.20.121.130:40271)
0929 16:09:26.810437 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:26.810444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:26.811149 (+ 705us) server_negotiation.cc:407] Connection header received
0929 16:09:26.811259 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:26.811269 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:26.811524 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:26.811709 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:26.812245 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.813683 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.817447 (+ 3764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:26.819986 (+ 2539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:26.820182 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:26.820329 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:26.820749 (+ 420us) server_negotiation.cc:299] Negotiation successful
0929 16:09:26.820924 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":402,"threads_started":1}
I20260929 16:09:26.822721 7220 tablet_service.cc:1467] Processing CreateTablet for tablet 43c111f8029e4562a99fb93077a025e4 (DEFAULT_TABLE table=test-table [id=22c717fdcb894510a0317a923929e7cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:26.823474 6952 tablet_service.cc:1467] Processing CreateTablet for tablet 43c111f8029e4562a99fb93077a025e4 (DEFAULT_TABLE table=test-table [id=22c717fdcb894510a0317a923929e7cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:26.824317 7220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43c111f8029e4562a99fb93077a025e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:26.824813 6952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43c111f8029e4562a99fb93077a025e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:26.824698 7086 tablet_service.cc:1467] Processing CreateTablet for tablet 43c111f8029e4562a99fb93077a025e4 (DEFAULT_TABLE table=test-table [id=22c717fdcb894510a0317a923929e7cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:26.825757 7086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43c111f8029e4562a99fb93077a025e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:26.838377 7304 tablet_bootstrap.cc:492] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: Bootstrap starting.
I20260929 16:09:26.840536 7306 tablet_bootstrap.cc:492] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: Bootstrap starting.
I20260929 16:09:26.842499 7304 tablet_bootstrap.cc:654] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:26.842795 7305 tablet_bootstrap.cc:492] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: Bootstrap starting.
I20260929 16:09:26.844013 7304 log.cc:824] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:26.844175 7306 tablet_bootstrap.cc:654] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:26.845679 7306 log.cc:824] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:26.846175 7305 tablet_bootstrap.cc:654] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:26.847289 7305 log.cc:824] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:26.850423 7305 tablet_bootstrap.cc:492] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: No bootstrap required, opened a new log
I20260929 16:09:26.850623 7304 tablet_bootstrap.cc:492] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: No bootstrap required, opened a new log
I20260929 16:09:26.850754 7305 ts_tablet_manager.cc:1397] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260929 16:09:26.850834 7306 tablet_bootstrap.cc:492] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: No bootstrap required, opened a new log
I20260929 16:09:26.850924 7304 ts_tablet_manager.cc:1397] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260929 16:09:26.851083 7306 ts_tablet_manager.cc:1397] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260929 16:09:26.858279 7305 raft_consensus.cc:348] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } }
I20260929 16:09:26.858721 7305 raft_consensus.cc:374] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:26.858819 7305 raft_consensus.cc:729] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2643340b254dcc9419de3d4d134706, State: Initialized, Role: FOLLOWER
I20260929 16:09:26.859543 7305 consensus_queue.cc:260] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } }
I20260929 16:09:26.861485 7305 ts_tablet_manager.cc:1428] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: Time spent starting tablet: real 0.011s user 0.005s sys 0.005s
I20260929 16:09:26.861738 7306 raft_consensus.cc:348] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } }
I20260929 16:09:26.862252 7306 raft_consensus.cc:374] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:26.862365 7306 raft_consensus.cc:729] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e07570e3e14ce0931d26cdb1b80649, State: Initialized, Role: FOLLOWER
I20260929 16:09:26.863008 7306 consensus_queue.cc:260] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } }
I20260929 16:09:26.864238 7285 heartbeater.cc:503] Master 127.20.121.190:38163 was elected leader, sending a full tablet report...
I20260929 16:09:26.863996 7304 raft_consensus.cc:348] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } }
I20260929 16:09:26.864503 7304 raft_consensus.cc:374] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:26.864596 7304 raft_consensus.cc:729] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f392c62849f4da9ae6c5429ef149170, State: Initialized, Role: FOLLOWER
I20260929 16:09:26.865082 7306 ts_tablet_manager.cc:1428] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260929 16:09:26.865439 7304 consensus_queue.cc:260] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } }
I20260929 16:09:26.866842 7151 heartbeater.cc:503] Master 127.20.121.190:38163 was elected leader, sending a full tablet report...
I20260929 16:09:26.867489 7304 ts_tablet_manager.cc:1428] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
W20260929 16:09:26.937002 7152 tablet.cc:2369] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:09:26.976325 7286 tablet.cc:2369] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:09:27.020277 7312 raft_consensus.cc:482] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:27.020774 7312 raft_consensus.cc:504] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } }
I20260929 16:09:27.022825 7312 leader_election.cc:288] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8e07570e3e14ce0931d26cdb1b80649 (127.20.121.131:45635), 0b2643340b254dcc9419de3d4d134706 (127.20.121.129:35071)
I20260929 16:09:27.026351 7311 raft_consensus.cc:482] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:27.026716 7311 raft_consensus.cc:504] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } }
I20260929 16:09:27.029017 7311 leader_election.cc:288] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b2643340b254dcc9419de3d4d134706 (127.20.121.129:35071), 7f392c62849f4da9ae6c5429ef149170 (127.20.121.130:40271)
I20260929 16:09:27.033282 7300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.023281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:34733 (local address 127.20.121.131:45635)
0929 16:09:27.023543 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:27.023552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:27.024649 (+ 1097us) server_negotiation.cc:407] Connection header received
0929 16:09:27.024809 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:27.024814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:27.024893 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:27.024988 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:27.025973 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.027216 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.029778 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.031586 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.031764 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.031890 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:27.032674 (+ 784us) server_negotiation.cc:299] Negotiation successful
0929 16:09:27.032891 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260929 16:09:27.034448 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.023301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:45635 (local address 127.20.121.130:34733)
0929 16:09:27.024521 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:27.024537 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:27.024668 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:27.025020 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:27.025026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:27.025060 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:27.025717 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:27.025728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.027335 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.027343 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:27.029566 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:27.029586 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.032148 (+ 2562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.032158 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:27.032422 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.032478 (+ 56us) client_negotiation.cc:769] Sending connection context
0929 16:09:27.033601 (+ 1123us) client_negotiation.cc:240] Negotiation successful
0929 16:09:27.033840 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":174,"thread_start_us":760,"threads_started":1}
I20260929 16:09:27.034796 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.023527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:42545 (local address 127.20.121.129:35071)
0929 16:09:27.023825 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:27.023835 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:27.024349 (+ 514us) server_negotiation.cc:407] Connection header received
0929 16:09:27.024628 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:27.024633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:27.024743 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:27.024855 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:27.025981 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.027115 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.030900 (+ 3785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.032794 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.032985 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.033097 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:27.034184 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0929 16:09:27.034414 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260929 16:09:27.037254 7240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c111f8029e4562a99fb93077a025e4" candidate_uuid: "7f392c62849f4da9ae6c5429ef149170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e07570e3e14ce0931d26cdb1b80649" is_pre_election: true
I20260929 16:09:27.037945 7240 raft_consensus.cc:2454] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f392c62849f4da9ae6c5429ef149170 in term 0.
I20260929 16:09:27.039331 7040 leader_election.cc:302] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f392c62849f4da9ae6c5429ef149170, d8e07570e3e14ce0931d26cdb1b80649; no voters:
I20260929 16:09:27.039971 7312 raft_consensus.cc:2781] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:09:27.040143 7312 raft_consensus.cc:482] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:09:27.040248 7312 raft_consensus.cc:3037] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:27.044171 7312 raft_consensus.cc:504] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } }
I20260929 16:09:27.045748 7288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.029382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35071 (local address 127.20.121.131:59529)
0929 16:09:27.029591 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:27.029617 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:27.029731 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:27.035601 (+ 5870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:27.035610 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:27.035661 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:27.035940 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:27.035955 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.038051 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.038060 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:27.040918 (+ 2858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:27.040944 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.044297 (+ 3353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.044306 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:27.044604 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.044684 (+ 80us) client_negotiation.cc:769] Sending connection context
0929 16:09:27.045250 (+ 566us) client_negotiation.cc:240] Negotiation successful
0929 16:09:27.045413 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 16:09:27.045965 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.029923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:59529 (local address 127.20.121.129:35071)
0929 16:09:27.035134 (+ 5211us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:27.035143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:27.035167 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:09:27.035244 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:27.035252 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:27.035377 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:27.035504 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:27.036187 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.037778 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.041266 (+ 3488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.043185 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.045402 (+ 2217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.045474 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:27.045529 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 16:09:27.045663 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5061,"thread_start_us":15212,"threads_started":1}
I20260929 16:09:27.046670 7240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c111f8029e4562a99fb93077a025e4" candidate_uuid: "7f392c62849f4da9ae6c5429ef149170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e07570e3e14ce0931d26cdb1b80649"
I20260929 16:09:27.046788 7312 leader_election.cc:288] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [CANDIDATE]: Term 1 election: Requested vote from peers d8e07570e3e14ce0931d26cdb1b80649 (127.20.121.131:45635), 0b2643340b254dcc9419de3d4d134706 (127.20.121.129:35071)
I20260929 16:09:27.046936 7240 raft_consensus.cc:3037] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:27.047421 7313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.023269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35071 (local address 127.20.121.130:42545)
0929 16:09:27.024204 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:27.024225 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:27.024436 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:27.025020 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:27.025029 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:27.025131 (+ 102us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:27.025670 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:27.025686 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.027280 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.027294 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:27.030596 (+ 3302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:27.030620 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.033347 (+ 2727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.033356 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:27.033870 (+ 514us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.033928 (+ 58us) client_negotiation.cc:769] Sending connection context
0929 16:09:27.046667 (+ 12739us) client_negotiation.cc:240] Negotiation successful
0929 16:09:27.046855 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":52,"spinlock_wait_cycles":27904,"thread_start_us":560,"threads_started":1}
I20260929 16:09:27.047616 6972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c111f8029e4562a99fb93077a025e4" candidate_uuid: "d8e07570e3e14ce0931d26cdb1b80649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2643340b254dcc9419de3d4d134706" is_pre_election: true
I20260929 16:09:27.048321 6972 raft_consensus.cc:2454] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8e07570e3e14ce0931d26cdb1b80649 in term 0.
I20260929 16:09:27.048471 6970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c111f8029e4562a99fb93077a025e4" candidate_uuid: "7f392c62849f4da9ae6c5429ef149170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2643340b254dcc9419de3d4d134706"
I20260929 16:09:27.048754 6970 raft_consensus.cc:3037] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:27.049384 7176 leader_election.cc:302] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2643340b254dcc9419de3d4d134706, d8e07570e3e14ce0931d26cdb1b80649; no voters:
I20260929 16:09:27.050414 6971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c111f8029e4562a99fb93077a025e4" candidate_uuid: "7f392c62849f4da9ae6c5429ef149170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2643340b254dcc9419de3d4d134706" is_pre_election: true
I20260929 16:09:27.050541 7240 raft_consensus.cc:2454] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f392c62849f4da9ae6c5429ef149170 in term 1.
I20260929 16:09:27.051173 7311 raft_consensus.cc:2741] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260929 16:09:27.051246 7040 leader_election.cc:302] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f392c62849f4da9ae6c5429ef149170, d8e07570e3e14ce0931d26cdb1b80649; no voters:
I20260929 16:09:27.052351 7312 raft_consensus.cc:2781] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:27.053336 7312 raft_consensus.cc:686] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 1 LEADER]: Becoming Leader. State: Replica: 7f392c62849f4da9ae6c5429ef149170, State: Running, Role: LEADER
I20260929 16:09:27.053747 6970 raft_consensus.cc:2454] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f392c62849f4da9ae6c5429ef149170 in term 1.
I20260929 16:09:27.054170 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.030233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:56209 (local address 127.20.121.130:40271)
0929 16:09:27.030478 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:27.030486 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:27.041888 (+ 11402us) server_negotiation.cc:407] Connection header received
0929 16:09:27.042401 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:27.042406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:27.042487 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:27.042549 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:27.043369 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.044845 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.051501 (+ 6656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.052858 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.053024 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.053102 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:27.053628 (+ 526us) server_negotiation.cc:299] Negotiation successful
0929 16:09:27.053786 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260929 16:09:27.053933 7312 consensus_queue.cc:237] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } }
I20260929 16:09:27.055088 7315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.029696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:40271 (local address 127.20.121.131:56209)
0929 16:09:27.041762 (+ 12066us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:27.041794 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:27.042227 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:27.042737 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:27.042744 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:27.042774 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:27.043178 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:27.043189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.048736 (+ 5547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.048744 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:27.050976 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:27.050994 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.053138 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.053148 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:27.053367 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.053404 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 16:09:27.054002 (+ 598us) client_negotiation.cc:240] Negotiation successful
0929 16:09:27.054704 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11923,"thread_start_us":12208,"threads_started":1}
I20260929 16:09:27.056121 7106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43c111f8029e4562a99fb93077a025e4" candidate_uuid: "d8e07570e3e14ce0931d26cdb1b80649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f392c62849f4da9ae6c5429ef149170" is_pre_election: true
I20260929 16:09:27.060225 6828 catalog_manager.cc:5495] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f392c62849f4da9ae6c5429ef149170 (127.20.121.130). New cstate: current_term: 1 leader_uuid: "7f392c62849f4da9ae6c5429ef149170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f392c62849f4da9ae6c5429ef149170" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 40271 } health_report { overall_health: HEALTHY } } }
W20260929 16:09:27.102639 7018 tablet.cc:2369] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:09:27.122223 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.116484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53158 (local address 127.20.121.130:40271)
0929 16:09:27.116699 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:27.116704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:27.116724 (+ 20us) server_negotiation.cc:407] Connection header received
0929 16:09:27.117022 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:27.117027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:27.117108 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:27.117242 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:09:27.117605 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.118394 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.119899 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.120298 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.120409 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.120937 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:09:27.121481 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:09:27.121620 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:27.121865 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 16:09:27.121913 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 16:09:27.131198 6970 raft_consensus.cc:1261] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706 [term 1 FOLLOWER]: Refusing update from remote peer 7f392c62849f4da9ae6c5429ef149170: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:27.131250 7240 raft_consensus.cc:1261] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 1 FOLLOWER]: Refusing update from remote peer 7f392c62849f4da9ae6c5429ef149170: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:27.132193 7317 consensus_queue.cc:1035] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8e07570e3e14ce0931d26cdb1b80649" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:27.132577 7312 consensus_queue.cc:1035] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b2643340b254dcc9419de3d4d134706" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:27.165271 6827 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:27.165630 6827 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:27.170501 6827 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53eac6c762c34cbf8deed65d5f7512c3: Sending DeleteTablet for 3 replicas of tablet 43c111f8029e4562a99fb93077a025e4
I20260929 16:09:27.172001 7220 tablet_service.cc:1514] Processing DeleteTablet for tablet 43c111f8029e4562a99fb93077a025e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:27 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:35870
I20260929 16:09:27.172349 6952 tablet_service.cc:1514] Processing DeleteTablet for tablet 43c111f8029e4562a99fb93077a025e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:27 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:52808
I20260929 16:09:27.173028 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 6889
I20260929 16:09:27.173206 7086 tablet_service.cc:1514] Processing DeleteTablet for tablet 43c111f8029e4562a99fb93077a025e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:27 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:53150
I20260929 16:09:27.203620 7332 tablet_replica.cc:330] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: stopping tablet replica
I20260929 16:09:27.204175 7332 raft_consensus.cc:2229] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:09:27.204917 7332 raft_consensus.cc:2258] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:27.205711 7331 tablet_replica.cc:330] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: stopping tablet replica
I20260929 16:09:27.206215 7331 raft_consensus.cc:2229] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:27.206784 7331 raft_consensus.cc:2258] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:27.210269 7331 ts_tablet_manager.cc:1905] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:27.212730 7332 ts_tablet_manager.cc:1905] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:27.218492 7330 tablet_replica.cc:330] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: stopping tablet replica
I20260929 16:09:27.219025 7330 raft_consensus.cc:2229] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:27.219636 7330 raft_consensus.cc:2258] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:27.220434 7331 ts_tablet_manager.cc:1918] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:27.220649 7331 log.cc:1196] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/43c111f8029e4562a99fb93077a025e4
I20260929 16:09:27.221179 7331 ts_tablet_manager.cc:1939] T 43c111f8029e4562a99fb93077a025e4 P 0b2643340b254dcc9419de3d4d134706: Deleting consensus metadata
I20260929 16:09:27.223126 6815 catalog_manager.cc:4841] TS 0b2643340b254dcc9419de3d4d134706 (127.20.121.129:35071): tablet 43c111f8029e4562a99fb93077a025e4 (table test-table [id=22c717fdcb894510a0317a923929e7cf]) successfully deleted
I20260929 16:09:27.229197 7330 ts_tablet_manager.cc:1905] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 16:09:27 dist-test-slave-x07d krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:09:27.238909 7330 ts_tablet_manager.cc:1918] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:27.239144 7330 log.cc:1196] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/43c111f8029e4562a99fb93077a025e4
I20260929 16:09:27.239786 7330 ts_tablet_manager.cc:1939] T 43c111f8029e4562a99fb93077a025e4 P d8e07570e3e14ce0931d26cdb1b80649: Deleting consensus metadata
I20260929 16:09:27.241906 7332 ts_tablet_manager.cc:1918] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:27.241906 6813 catalog_manager.cc:4841] TS d8e07570e3e14ce0931d26cdb1b80649 (127.20.121.131:45635): tablet 43c111f8029e4562a99fb93077a025e4 (table test-table [id=22c717fdcb894510a0317a923929e7cf]) successfully deleted
I20260929 16:09:27.242264 7332 log.cc:1196] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/43c111f8029e4562a99fb93077a025e4
I20260929 16:09:27.242818 7332 ts_tablet_manager.cc:1939] T 43c111f8029e4562a99fb93077a025e4 P 7f392c62849f4da9ae6c5429ef149170: Deleting consensus metadata
I20260929 16:09:27.242866 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.174314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52810 (local address 127.20.121.129:35071)
0929 16:09:27.174509 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:27.174515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:27.196431 (+ 21916us) server_negotiation.cc:407] Connection header received
0929 16:09:27.200926 (+ 4495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:27.200934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:27.201034 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:27.201115 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:27.207280 (+ 6165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.208478 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.226229 (+ 17751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.227069 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.227272 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.233939 (+ 6667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:27.233961 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:27.233984 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:27.234076 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:27.238733 (+ 4657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:27.239795 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:27.239806 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:27.239810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:27.240009 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:27.240487 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:27.240493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:27.240497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:27.240981 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:27.241318 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:27.242233 (+ 915us) server_negotiation.cc:299] Negotiation successful
0929 16:09:27.242495 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 16:09:27.244169 7012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:27.244846 6813 catalog_manager.cc:4841] TS 7f392c62849f4da9ae6c5429ef149170 (127.20.121.130:40271): tablet 43c111f8029e4562a99fb93077a025e4 (table test-table [id=22c717fdcb894510a0317a923929e7cf]) successfully deleted
I20260929 16:09:27.339987 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 6889
I20260929 16:09:27.358721 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 7023
Sep 29 16:09:27 dist-test-slave-x07d krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:09:27.378451 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.359588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53164 (local address 127.20.121.130:40271)
0929 16:09:27.359851 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:27.359855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:27.359878 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:09:27.360163 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:27.360168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:27.360244 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:27.360389 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:27.362144 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.362933 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.364776 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.365266 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.365366 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.370917 (+ 5551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:27.370936 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:27.370957 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:27.371016 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:27.375231 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:27.376099 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:27.376108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:27.376113 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:27.376287 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:27.376663 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:27.376668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:27.376673 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:27.377164 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:27.377492 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:27.377775 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 16:09:27.378082 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260929 16:09:27.379317 7146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:27.498533 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 7023
I20260929 16:09:27.520685 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 7157
Sep 29 16:09:27 dist-test-slave-x07d krb5kdc[6783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:09:27.537907 7300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:27.521502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.20.121.131:45635)
0929 16:09:27.521701 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:27.521710 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:27.521733 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:09:27.522085 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:27.522089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:27.522157 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:27.522307 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:27.523587 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.524344 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.525708 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:27.526334 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:27.526489 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:27.531542 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:27.531559 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:27.531576 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:27.531621 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:27.534673 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:27.535599 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:27.535608 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:27.535610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:27.535841 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:27.536245 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:27.536249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:27.536251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:27.536646 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:27.536918 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:27.537248 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 16:09:27.537433 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 16:09:27.538622 7280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:27.631503 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 7157
I20260929 16:09:27.648653 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 6796
I20260929 16:09:27.650141 6858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:27.762979 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 6796
2026-09-29T16:09:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4740 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T16:09:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:27Z Disabled control of system clock
I20260929 16:09:27.846140 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:35939
--webserver_interface=127.20.121.190
--webserver_port=0
--builtin_ntp_servers=127.20.121.148:35527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:35939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 16:09:28.216339 7341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:28.216652 7341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:28.216735 7341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:28.224232 7341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:28.224359 7341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:28.224397 7341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:28.224447 7341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:28.224491 7341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:28.234478 7341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:35939
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:35939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:28.236030 7341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:28.237960 7341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:28.250730 7346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:28.250845 7347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:28.250895 7349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:28.252211 7341 server_base.cc:1034] running on GCE node
I20260929 16:09:28.253120 7341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:28.254822 7341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:28.256067 7341 hybrid_clock.cc:648] HybridClock initialized: now 1790698168256012 us; error 74 us; skew 500 ppm
I20260929 16:09:28.256443 7341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:28.259022 7341 webserver.cc:460] Webserver started at http://127.20.121.190:46049/ using document root <none> and password file <none>
I20260929 16:09:28.259796 7341 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:28.259940 7341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:28.260281 7341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:28.263604 7341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "4c7ec1a6c24d48d38a79f99cd72082ba"
format_stamp: "Formatted at 2026-09-29 16:09:28 on dist-test-slave-x07d"
server_key: "74ab9bf3d7a90b710880ef04c396860a"
server_key_iv: "1b63ad9b19bd0dcddfc5772060b9956f"
server_key_version: "encryptionkey@0"
I20260929 16:09:28.264515 7341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "4c7ec1a6c24d48d38a79f99cd72082ba"
format_stamp: "Formatted at 2026-09-29 16:09:28 on dist-test-slave-x07d"
server_key: "74ab9bf3d7a90b710880ef04c396860a"
server_key_iv: "1b63ad9b19bd0dcddfc5772060b9956f"
server_key_version: "encryptionkey@0"
I20260929 16:09:28.270205 7341 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 16:09:28.274152 7355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:28.275660 7341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:09:28.275897 7341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "4c7ec1a6c24d48d38a79f99cd72082ba"
format_stamp: "Formatted at 2026-09-29 16:09:28 on dist-test-slave-x07d"
server_key: "74ab9bf3d7a90b710880ef04c396860a"
server_key_iv: "1b63ad9b19bd0dcddfc5772060b9956f"
server_key_version: "encryptionkey@0"
I20260929 16:09:28.276155 7341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:28.290892 7341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:28.291817 7341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:28.292155 7341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:28.312098 7341 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:35939
I20260929 16:09:28.312274 7406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:35939 every 8 connection(s)
I20260929 16:09:28.313524 7341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:28.317634 7407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:28.323002 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 7341
I20260929 16:09:28.323216 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:28.323800 20966 external_mini_cluster.cc:1428] Setting key 5e81b1d9fd83215b22aac52ee9bcac20
I20260929 16:09:28.331480 7407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba: Bootstrap starting.
I20260929 16:09:28.335618 7407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:28.337108 7407 log.cc:824] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:28.340937 7407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba: No bootstrap required, opened a new log
I20260929 16:09:28.341341 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:28.326000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52062 (local address 127.20.121.190:35939)
0929 16:09:28.327277 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:28.327294 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:28.327354 (+ 60us) server_negotiation.cc:407] Connection header received
0929 16:09:28.330100 (+ 2746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:28.330132 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:28.330775 (+ 643us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:28.331273 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:28.331854 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:28.333982 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:28.335658 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:28.336804 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:28.337024 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:28.338467 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:28.338478 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:28.338493 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:28.339221 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:28.339322 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:28.339368 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:28.339503 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:28.339686 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 16:09:28.340034 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":352,"threads_started":1}
I20260929 16:09:28.349475 7407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ec1a6c24d48d38a79f99cd72082ba" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35939 } }
I20260929 16:09:28.349901 7407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:28.349980 7407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7ec1a6c24d48d38a79f99cd72082ba, State: Initialized, Role: FOLLOWER
I20260929 16:09:28.350657 7407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ec1a6c24d48d38a79f99cd72082ba" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35939 } }
I20260929 16:09:28.350889 7407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:28.350998 7407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:28.351126 7407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:28.353614 7407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ec1a6c24d48d38a79f99cd72082ba" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35939 } }
I20260929 16:09:28.354183 7407 leader_election.cc:302] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c7ec1a6c24d48d38a79f99cd72082ba; no voters:
I20260929 16:09:28.354822 7407 leader_election.cc:288] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:28.355028 7412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:28.355819 7412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [term 1 LEADER]: Becoming Leader. State: Replica: 4c7ec1a6c24d48d38a79f99cd72082ba, State: Running, Role: LEADER
I20260929 16:09:28.356348 7412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ec1a6c24d48d38a79f99cd72082ba" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35939 } }
I20260929 16:09:28.357088 7407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:28.359469 7414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c7ec1a6c24d48d38a79f99cd72082ba. Latest consensus state: current_term: 1 leader_uuid: "4c7ec1a6c24d48d38a79f99cd72082ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ec1a6c24d48d38a79f99cd72082ba" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35939 } } }
I20260929 16:09:28.359663 7413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c7ec1a6c24d48d38a79f99cd72082ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ec1a6c24d48d38a79f99cd72082ba" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35939 } } }
I20260929 16:09:28.359982 7413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:28.359997 7414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:28.363655 7420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:28.369025 7420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:28.378898 7420 catalog_manager.cc:1347] Generated new cluster ID: 8267efe7aef14d6eb9d028cd9e68343c
I20260929 16:09:28.379108 7420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:28.391716 7420 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:28.393049 7420 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:28.405692 7420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba: Generated new TSK 0
I20260929 16:09:28.406668 7420 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:28.416874 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35939
--builtin_ntp_servers=127.20.121.148:35527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 16:09:28.772113 7431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:28.772416 7431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:28.772570 7431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:28.780380 7431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:28.780499 7431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:28.780671 7431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:09:28.790360 7431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:28.791742 7431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:28.793588 7431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:28.808629 7436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:28.808604 7439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:28.808696 7437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:28.809580 7431 server_base.cc:1034] running on GCE node
I20260929 16:09:28.810418 7431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:28.811496 7431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:28.812790 7431 hybrid_clock.cc:648] HybridClock initialized: now 1790698168812724 us; error 56 us; skew 500 ppm
I20260929 16:09:28.813295 7431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:28.816310 7431 webserver.cc:460] Webserver started at http://127.20.121.129:33141/ using document root <none> and password file <none>
I20260929 16:09:28.817155 7431 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:28.817322 7431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:28.817705 7431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:28.821259 7431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "3399676174d540a9a8e54656e02cd0ab"
format_stamp: "Formatted at 2026-09-29 16:09:28 on dist-test-slave-x07d"
server_key: "2ff0a5168aaacf4d44a1790a5b419217"
server_key_iv: "584d66918be5110aa3ec0382989129b7"
server_key_version: "encryptionkey@0"
I20260929 16:09:28.822438 7431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "3399676174d540a9a8e54656e02cd0ab"
format_stamp: "Formatted at 2026-09-29 16:09:28 on dist-test-slave-x07d"
server_key: "2ff0a5168aaacf4d44a1790a5b419217"
server_key_iv: "584d66918be5110aa3ec0382989129b7"
server_key_version: "encryptionkey@0"
I20260929 16:09:28.829071 7431 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 16:09:28.834072 7445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:28.835884 7431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:09:28.836107 7431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "3399676174d540a9a8e54656e02cd0ab"
format_stamp: "Formatted at 2026-09-29 16:09:28 on dist-test-slave-x07d"
server_key: "2ff0a5168aaacf4d44a1790a5b419217"
server_key_iv: "584d66918be5110aa3ec0382989129b7"
server_key_version: "encryptionkey@0"
I20260929 16:09:28.836354 7431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:28.851326 7431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:28.852246 7431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:28.852572 7431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:28.853646 7431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:28.855407 7431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:28.855504 7431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:28.855659 7431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:28.855741 7431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:28.892196 7431 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:40259
I20260929 16:09:28.892235 7557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:40259 every 8 connection(s)
I20260929 16:09:28.894059 7431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:09:28.904146 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 7431
I20260929 16:09:28.904353 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:09:28.904911 20966 external_mini_cluster.cc:1428] Setting key 05da8f3ca080e5676e8b5320716bb83d
I20260929 16:09:28.908569 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35939
--builtin_ntp_servers=127.20.121.148:35527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 16:09:28.908881 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:28.896946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:42309 (local address 127.20.121.190:35939)
0929 16:09:28.897509 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:28.897514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:28.898678 (+ 1164us) server_negotiation.cc:407] Connection header received
0929 16:09:28.900805 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:28.900810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:28.900907 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:28.901029 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:28.902228 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:28.902946 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:28.905112 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:28.905644 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:28.905827 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:28.907683 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:28.907690 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:28.907695 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:28.907718 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:28.907869 (+ 151us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:28.907891 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:28.907992 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:28.908331 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 16:09:28.908441 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":335,"threads_started":1}
I20260929 16:09:28.910118 7561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:28.897433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:35939 (local address 127.20.121.129:42309)
0929 16:09:28.898512 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:28.898555 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:28.900492 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:28.901294 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:28.901304 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:28.901401 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:28.901995 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:28.902070 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:28.903108 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:28.903120 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:28.904905 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:28.904926 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:28.905935 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:28.905943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:28.906290 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:28.907249 (+ 959us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:09:28.907253 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:28.907317 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:09:28.907440 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:28.908033 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:28.908049 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:28.908061 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 16:09:28.908273 (+ 212us) client_negotiation.cc:240] Negotiation successful
0929 16:09:28.908772 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":355,"threads_started":1}
I20260929 16:09:28.911880 7558 heartbeater.cc:347] Connected to a master server at 127.20.121.190:35939
I20260929 16:09:28.912207 7558 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:28.913038 7558 heartbeater.cc:511] Master 127.20.121.190:35939 requested a full tablet report, sending...
I20260929 16:09:28.915217 7372 ts_manager.cc:194] Registered new tserver with Master: 3399676174d540a9a8e54656e02cd0ab (127.20.121.129:40259)
I20260929 16:09:28.916944 7372 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.129:42309
W20260929 16:09:29.257561 7562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:29.257864 7562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:29.258092 7562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:29.266273 7562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:29.266402 7562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:29.266580 7562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:09:29.276618 7562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:29.278041 7562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:29.280049 7562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:29.294638 7567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:29.294685 7570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:29.294638 7568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:29.295830 7562 server_base.cc:1034] running on GCE node
I20260929 16:09:29.296672 7562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:29.297725 7562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:29.298964 7562 hybrid_clock.cc:648] HybridClock initialized: now 1790698169298903 us; error 52 us; skew 500 ppm
I20260929 16:09:29.299329 7562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:29.302057 7562 webserver.cc:460] Webserver started at http://127.20.121.130:36303/ using document root <none> and password file <none>
I20260929 16:09:29.302773 7562 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:29.302867 7562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:29.303140 7562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:29.307602 7562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "03462ddcfc5e456885ba66d1939621cc"
format_stamp: "Formatted at 2026-09-29 16:09:29 on dist-test-slave-x07d"
server_key: "23c3621687261297b473fd91a29b7d6a"
server_key_iv: "59cef6317ee4b0ef8759b1e235152e6e"
server_key_version: "encryptionkey@0"
I20260929 16:09:29.308897 7562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "03462ddcfc5e456885ba66d1939621cc"
format_stamp: "Formatted at 2026-09-29 16:09:29 on dist-test-slave-x07d"
server_key: "23c3621687261297b473fd91a29b7d6a"
server_key_iv: "59cef6317ee4b0ef8759b1e235152e6e"
server_key_version: "encryptionkey@0"
I20260929 16:09:29.318797 7562 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260929 16:09:29.323099 7576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:29.324769 7562 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260929 16:09:29.324991 7562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "03462ddcfc5e456885ba66d1939621cc"
format_stamp: "Formatted at 2026-09-29 16:09:29 on dist-test-slave-x07d"
server_key: "23c3621687261297b473fd91a29b7d6a"
server_key_iv: "59cef6317ee4b0ef8759b1e235152e6e"
server_key_version: "encryptionkey@0"
I20260929 16:09:29.325265 7562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:29.361490 7562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:29.362581 7562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:29.362936 7562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:29.364084 7562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:29.365980 7562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:29.366178 7562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260929 16:09:29.366341 7562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:29.366441 7562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:29.403954 7562 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:41743
I20260929 16:09:29.404058 7688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:41743 every 8 connection(s)
I20260929 16:09:29.405364 7562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:09:29.406564 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 7562
I20260929 16:09:29.406771 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:09:29.407233 20966 external_mini_cluster.cc:1428] Setting key 09e9483cad0c38bd9e59d7bb88b15740
I20260929 16:09:29.410658 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35939
--builtin_ntp_servers=127.20.121.148:35527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 16:09:29.421967 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.408365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:52299 (local address 127.20.121.190:35939)
0929 16:09:29.408572 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:29.408577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:29.410426 (+ 1849us) server_negotiation.cc:407] Connection header received
0929 16:09:29.413390 (+ 2964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:29.413394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:29.413466 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:29.413562 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:29.415185 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.415922 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.418720 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.419235 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.419375 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.421117 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:29.421122 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:29.421126 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:29.421150 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:29.421183 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:29.421196 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:29.421266 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:29.421572 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 16:09:29.421667 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:09:29.423540 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.409056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:35939 (local address 127.20.121.130:52299)
0929 16:09:29.410230 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:29.410293 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:29.413028 (+ 2735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:29.413870 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:29.413886 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:29.414051 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:29.414944 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:29.414973 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.416104 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.416113 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:29.418510 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:29.418532 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.419541 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.419551 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:29.419874 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.420697 (+ 823us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:09:29.420700 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:29.420744 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:09:29.420844 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:29.421296 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:29.421310 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:29.421322 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 16:09:29.421519 (+ 197us) client_negotiation.cc:240] Negotiation successful
0929 16:09:29.421986 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":337,"threads_started":1}
I20260929 16:09:29.425326 7689 heartbeater.cc:347] Connected to a master server at 127.20.121.190:35939
I20260929 16:09:29.425660 7689 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:29.426551 7689 heartbeater.cc:511] Master 127.20.121.190:35939 requested a full tablet report, sending...
I20260929 16:09:29.428170 7372 ts_manager.cc:194] Registered new tserver with Master: 03462ddcfc5e456885ba66d1939621cc (127.20.121.130:41743)
I20260929 16:09:29.429212 7372 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.130:52299
W20260929 16:09:29.763463 7692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:29.763813 7692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:29.763968 7692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:29.771528 7692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:29.771653 7692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:29.771848 7692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:09:29.781656 7692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:35939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:29.783070 7692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:29.784924 7692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:29.799469 7697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:29.799515 7698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:29.799556 7700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:29.800491 7692 server_base.cc:1034] running on GCE node
I20260929 16:09:29.801223 7692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:29.802386 7692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:29.803665 7692 hybrid_clock.cc:648] HybridClock initialized: now 1790698169803592 us; error 61 us; skew 500 ppm
I20260929 16:09:29.804071 7692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:29.806665 7692 webserver.cc:460] Webserver started at http://127.20.121.131:36837/ using document root <none> and password file <none>
I20260929 16:09:29.807425 7692 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:29.807561 7692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:29.807895 7692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:29.811280 7692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "2ce24bd88bb940c7b2620b0662af02de"
format_stamp: "Formatted at 2026-09-29 16:09:29 on dist-test-slave-x07d"
server_key: "16ec68822e2efa8f4da09b4d48e79586"
server_key_iv: "e8689f8beb120b0d2a079f62f4e9585e"
server_key_version: "encryptionkey@0"
I20260929 16:09:29.812160 7692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "2ce24bd88bb940c7b2620b0662af02de"
format_stamp: "Formatted at 2026-09-29 16:09:29 on dist-test-slave-x07d"
server_key: "16ec68822e2efa8f4da09b4d48e79586"
server_key_iv: "e8689f8beb120b0d2a079f62f4e9585e"
server_key_version: "encryptionkey@0"
I20260929 16:09:29.818425 7692 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 16:09:29.822958 7706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:29.824589 7692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:09:29.824841 7692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "2ce24bd88bb940c7b2620b0662af02de"
format_stamp: "Formatted at 2026-09-29 16:09:29 on dist-test-slave-x07d"
server_key: "16ec68822e2efa8f4da09b4d48e79586"
server_key_iv: "e8689f8beb120b0d2a079f62f4e9585e"
server_key_version: "encryptionkey@0"
I20260929 16:09:29.825124 7692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:29.845069 7692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:29.845983 7692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:29.846356 7692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:29.847462 7692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:29.849246 7692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:29.849382 7692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:29.849512 7692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:29.849611 7692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:29.885824 7692 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:45947
I20260929 16:09:29.885923 7818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:45947 every 8 connection(s)
I20260929 16:09:29.887329 7692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:09:29.892755 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 7692
I20260929 16:09:29.893090 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:09:29.893564 20966 external_mini_cluster.cc:1428] Setting key 3cc642a80404d0a5678ab16762cdbfac
I20260929 16:09:29.909106 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.890347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:47491 (local address 127.20.121.190:35939)
0929 16:09:29.890566 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:29.890572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:29.892563 (+ 1991us) server_negotiation.cc:407] Connection header received
0929 16:09:29.895481 (+ 2918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:29.895486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:29.895592 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:29.895705 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:29.897448 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.898386 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.900190 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.900625 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.900803 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.907904 (+ 7101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:29.907912 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:29.907915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:29.907938 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:29.908005 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:29.908029 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:29.908137 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:29.908504 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 16:09:29.908659 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 16:09:29.910496 7821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.890948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:35939 (local address 127.20.121.131:47491)
0929 16:09:29.892330 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:29.892386 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:29.895103 (+ 2717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:29.896034 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:29.896048 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:29.896197 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:29.897149 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:29.897174 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.898558 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.898564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:29.900005 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:29.900021 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.900963 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.900969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:29.901182 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.907413 (+ 6231us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:09:29.907418 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:29.907493 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:09:29.907618 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:29.908188 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:29.908205 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:29.908215 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 16:09:29.908449 (+ 234us) client_negotiation.cc:240] Negotiation successful
0929 16:09:29.909106 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":382,"threads_started":1}
I20260929 16:09:29.912029 7819 heartbeater.cc:347] Connected to a master server at 127.20.121.190:35939
I20260929 16:09:29.912333 7819 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:29.913154 7819 heartbeater.cc:511] Master 127.20.121.190:35939 requested a full tablet report, sending...
I20260929 16:09:29.914700 7372 ts_manager.cc:194] Registered new tserver with Master: 2ce24bd88bb940c7b2620b0662af02de (127.20.121.131:45947)
I20260929 16:09:29.915580 7372 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.131:47491
I20260929 16:09:29.920104 7558 heartbeater.cc:503] Master 127.20.121.190:35939 was elected leader, sending a full tablet report...
I20260929 16:09:29.922474 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:09:29.933944 7560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.926958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52074 (local address 127.20.121.190:35939)
0929 16:09:29.927156 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:29.927160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:29.927597 (+ 437us) server_negotiation.cc:407] Connection header received
0929 16:09:29.927777 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:29.927782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:29.927853 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:29.927948 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:29.928447 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.929388 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.931363 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.931852 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.931977 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.933218 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:29.933223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:29.933227 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:29.933258 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:29.933293 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:29.933306 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:29.933431 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:29.933612 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 16:09:29.933664 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 16:09:29.939311 7372 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:09:29.946604 7372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:09:29.974380 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.961238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:45947 (local address 127.0.0.1:49996)
0929 16:09:29.963551 (+ 2313us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:29.963607 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:29.963788 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:29.965611 (+ 1823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:29.965618 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:29.965651 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:29.966305 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:29.966324 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.968431 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.968436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:29.971314 (+ 2878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:29.971327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.973335 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.973342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:29.973567 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.973629 (+ 62us) client_negotiation.cc:769] Sending connection context
0929 16:09:29.973741 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 16:09:29.973940 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2030,"mutex_wait_us":26,"thread_start_us":2373,"threads_started":1}
I20260929 16:09:29.975741 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.963426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.20.121.130:41743)
0929 16:09:29.965702 (+ 2276us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:29.965712 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:29.966995 (+ 1283us) server_negotiation.cc:407] Connection header received
0929 16:09:29.967365 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:29.967375 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:29.967646 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:29.967859 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:29.969282 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.970846 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.972820 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.974212 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.974373 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.974619 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:29.975212 (+ 593us) server_negotiation.cc:299] Negotiation successful
0929 16:09:29.975393 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2103,"thread_start_us":774,"threads_started":1}
I20260929 16:09:29.975726 7836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.963388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:41743 (local address 127.0.0.1:56040)
0929 16:09:29.966873 (+ 3485us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:29.966891 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:29.967105 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:29.968133 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:29.968137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:29.968156 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:29.969068 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:29.969084 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.971034 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.971039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:29.972625 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:29.972637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.974758 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.974764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:29.974927 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.974952 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 16:09:29.975165 (+ 213us) client_negotiation.cc:240] Negotiation successful
0929 16:09:29.975316 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3365,"mutex_wait_us":104,"thread_start_us":3215,"threads_started":1}
I20260929 16:09:29.975886 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.961876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49996 (local address 127.20.121.131:45947)
0929 16:09:29.964895 (+ 3019us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:29.964904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:29.964941 (+ 37us) server_negotiation.cc:407] Connection header received
0929 16:09:29.965039 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:29.965046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:29.965331 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:29.966552 (+ 1221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:29.966889 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.968258 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.971552 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.973107 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.974978 (+ 1871us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.975241 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:29.975321 (+ 80us) server_negotiation.cc:299] Negotiation successful
0929 16:09:29.975495 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2838,"thread_start_us":2664,"threads_started":1}
I20260929 16:09:29.976475 7832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.961254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40259 (local address 127.0.0.1:34678)
0929 16:09:29.966245 (+ 4991us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:29.966271 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:29.966395 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:29.967872 (+ 1477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:29.967877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:29.967903 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:29.969100 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:29.969116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.970777 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.970783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:29.972898 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:29.972913 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.975611 (+ 2698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.975618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:29.975815 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.975860 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 16:09:29.975956 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 16:09:29.976154 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4818,"mutex_wait_us":146,"thread_start_us":4629,"threads_started":1}
I20260929 16:09:29.977289 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:29.961382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34678 (local address 127.20.121.129:40259)
0929 16:09:29.967291 (+ 5909us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:29.967300 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:29.967337 (+ 37us) server_negotiation.cc:407] Connection header received
0929 16:09:29.967423 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:29.967430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:29.967647 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:29.967856 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:29.969342 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.970630 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.973126 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:29.975079 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:29.975282 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:29.975472 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:29.976719 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0929 16:09:29.976901 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5707,"thread_start_us":5569,"threads_started":1}
I20260929 16:09:29.980049 7754 tablet_service.cc:1467] Processing CreateTablet for tablet 84acb42475df4eb381e9e7578fe5cc1d (DEFAULT_TABLE table=test-table [id=bd0c3ae547284c78b3a789e498da031f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:29.980053 7624 tablet_service.cc:1467] Processing CreateTablet for tablet 84acb42475df4eb381e9e7578fe5cc1d (DEFAULT_TABLE table=test-table [id=bd0c3ae547284c78b3a789e498da031f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:29.981392 7624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84acb42475df4eb381e9e7578fe5cc1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:29.981398 7754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84acb42475df4eb381e9e7578fe5cc1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:29.988188 7493 tablet_service.cc:1467] Processing CreateTablet for tablet 84acb42475df4eb381e9e7578fe5cc1d (DEFAULT_TABLE table=test-table [id=bd0c3ae547284c78b3a789e498da031f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:29.989888 7493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84acb42475df4eb381e9e7578fe5cc1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:29.995837 7837 tablet_bootstrap.cc:492] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: Bootstrap starting.
I20260929 16:09:29.999646 7837 tablet_bootstrap.cc:654] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:30.000799 7837 log.cc:824] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:30.001906 7838 tablet_bootstrap.cc:492] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: Bootstrap starting.
I20260929 16:09:30.007387 7838 tablet_bootstrap.cc:654] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:30.008205 7840 tablet_bootstrap.cc:492] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab: Bootstrap starting.
I20260929 16:09:30.008810 7838 log.cc:824] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:30.009281 7837 tablet_bootstrap.cc:492] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: No bootstrap required, opened a new log
I20260929 16:09:30.009646 7837 ts_tablet_manager.cc:1397] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260929 16:09:30.011879 7840 tablet_bootstrap.cc:654] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:30.012529 7838 tablet_bootstrap.cc:492] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: No bootstrap required, opened a new log
I20260929 16:09:30.012871 7838 ts_tablet_manager.cc:1397] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260929 16:09:30.013072 7840 log.cc:824] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:30.017127 7840 tablet_bootstrap.cc:492] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab: No bootstrap required, opened a new log
I20260929 16:09:30.017481 7840 ts_tablet_manager.cc:1397] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260929 16:09:30.021782 7837 raft_consensus.cc:348] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 } } peers { permanent_uuid: "2ce24bd88bb940c7b2620b0662af02de" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45947 } } peers { permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 } }
I20260929 16:09:30.022387 7837 raft_consensus.cc:374] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:30.022506 7837 raft_consensus.cc:729] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ce24bd88bb940c7b2620b0662af02de, State: Initialized, Role: FOLLOWER
I20260929 16:09:30.023260 7837 consensus_queue.cc:260] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 } } peers { permanent_uuid: "2ce24bd88bb940c7b2620b0662af02de" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45947 } } peers { permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 } }
I20260929 16:09:30.024791 7819 heartbeater.cc:503] Master 127.20.121.190:35939 was elected leader, sending a full tablet report...
I20260929 16:09:30.024484 7838 raft_consensus.cc:348] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 } } peers { permanent_uuid: "2ce24bd88bb940c7b2620b0662af02de" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45947 } } peers { permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 } }
I20260929 16:09:30.025010 7838 raft_consensus.cc:374] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:30.025117 7838 raft_consensus.cc:729] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03462ddcfc5e456885ba66d1939621cc, State: Initialized, Role: FOLLOWER
I20260929 16:09:30.025782 7838 consensus_queue.cc:260] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 } } peers { permanent_uuid: "2ce24bd88bb940c7b2620b0662af02de" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45947 } } peers { permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 } }
I20260929 16:09:30.026723 7840 raft_consensus.cc:348] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 } } peers { permanent_uuid: "2ce24bd88bb940c7b2620b0662af02de" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45947 } } peers { permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 } }
I20260929 16:09:30.027172 7837 ts_tablet_manager.cc:1428] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260929 16:09:30.027209 7840 raft_consensus.cc:374] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:30.027302 7840 raft_consensus.cc:729] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3399676174d540a9a8e54656e02cd0ab, State: Initialized, Role: FOLLOWER
I20260929 16:09:30.027944 7840 consensus_queue.cc:260] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 } } peers { permanent_uuid: "2ce24bd88bb940c7b2620b0662af02de" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45947 } } peers { permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 } }
I20260929 16:09:30.029246 7838 ts_tablet_manager.cc:1428] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260929 16:09:30.029512 7689 heartbeater.cc:503] Master 127.20.121.190:35939 was elected leader, sending a full tablet report...
I20260929 16:09:30.031116 7840 ts_tablet_manager.cc:1428] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
W20260929 16:09:30.139317 7820 tablet.cc:2369] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:09:30.146971 7559 tablet.cc:2369] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:09:30.157811 7690 tablet.cc:2369] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:09:30.233343 7843 raft_consensus.cc:482] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:30.233628 7843 raft_consensus.cc:504] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 } } peers { permanent_uuid: "2ce24bd88bb940c7b2620b0662af02de" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45947 } } peers { permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 } }
I20260929 16:09:30.235416 7843 leader_election.cc:288] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3399676174d540a9a8e54656e02cd0ab (127.20.121.129:40259), 03462ddcfc5e456885ba66d1939621cc (127.20.121.130:41743)
I20260929 16:09:30.242184 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.236072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:47821 (local address 127.20.121.129:40259)
0929 16:09:30.236268 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:30.236273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:30.236448 (+ 175us) server_negotiation.cc:407] Connection header received
0929 16:09:30.236627 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:30.236631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:30.236703 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:30.236793 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:30.237422 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.238547 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.240299 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.241242 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.241423 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:30.241496 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:30.241761 (+ 265us) server_negotiation.cc:299] Negotiation successful
0929 16:09:30.241900 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 16:09:30.242228 7821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.236131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40259 (local address 127.20.121.131:47821)
0929 16:09:30.236308 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:30.236356 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:30.236502 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:30.236894 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:30.236904 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:30.236951 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:30.237228 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:30.237240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.238679 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.238685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:30.240117 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:30.240130 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.241423 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.241429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:30.241597 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:30.241623 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 16:09:30.241731 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 16:09:30.241879 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":3}
I20260929 16:09:30.242908 7846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.236130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:41743 (local address 127.20.121.131:49881)
0929 16:09:30.237378 (+ 1248us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:30.237398 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:30.237537 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:30.237908 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:30.237913 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:30.237938 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:30.238344 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:30.238364 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.239442 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.239448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:30.240933 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:30.240946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.242167 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.242173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:30.242382 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:30.242412 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 16:09:30.242504 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 16:09:30.242626 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1099,"mutex_wait_us":452,"thread_start_us":456,"threads_started":1}
I20260929 16:09:30.242985 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.236013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:49881 (local address 127.20.121.130:41743)
0929 16:09:30.236227 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:30.236234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:30.237511 (+ 1277us) server_negotiation.cc:407] Connection header received
0929 16:09:30.237678 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:30.237682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:30.237754 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:30.237890 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:30.238523 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.239247 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.241115 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.241973 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.242166 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:30.242235 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:30.242546 (+ 311us) server_negotiation.cc:299] Negotiation successful
0929 16:09:30.242700 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 16:09:30.243609 7513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84acb42475df4eb381e9e7578fe5cc1d" candidate_uuid: "2ce24bd88bb940c7b2620b0662af02de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3399676174d540a9a8e54656e02cd0ab" is_pre_election: true
I20260929 16:09:30.244000 7644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84acb42475df4eb381e9e7578fe5cc1d" candidate_uuid: "2ce24bd88bb940c7b2620b0662af02de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03462ddcfc5e456885ba66d1939621cc" is_pre_election: true
I20260929 16:09:30.244148 7513 raft_consensus.cc:2454] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ce24bd88bb940c7b2620b0662af02de in term 0.
I20260929 16:09:30.244498 7644 raft_consensus.cc:2454] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ce24bd88bb940c7b2620b0662af02de in term 0.
I20260929 16:09:30.244953 7709 leader_election.cc:302] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ce24bd88bb940c7b2620b0662af02de, 3399676174d540a9a8e54656e02cd0ab; no voters:
I20260929 16:09:30.245452 7843 raft_consensus.cc:2781] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:09:30.245617 7843 raft_consensus.cc:482] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:09:30.245729 7843 raft_consensus.cc:3037] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:30.247988 7843 raft_consensus.cc:504] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 } } peers { permanent_uuid: "2ce24bd88bb940c7b2620b0662af02de" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45947 } } peers { permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 } }
I20260929 16:09:30.248642 7843 leader_election.cc:288] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [CANDIDATE]: Term 1 election: Requested vote from peers 3399676174d540a9a8e54656e02cd0ab (127.20.121.129:40259), 03462ddcfc5e456885ba66d1939621cc (127.20.121.130:41743)
I20260929 16:09:30.249238 7644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84acb42475df4eb381e9e7578fe5cc1d" candidate_uuid: "2ce24bd88bb940c7b2620b0662af02de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03462ddcfc5e456885ba66d1939621cc"
I20260929 16:09:30.249230 7513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84acb42475df4eb381e9e7578fe5cc1d" candidate_uuid: "2ce24bd88bb940c7b2620b0662af02de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3399676174d540a9a8e54656e02cd0ab"
I20260929 16:09:30.249490 7513 raft_consensus.cc:3037] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:30.249490 7644 raft_consensus.cc:3037] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:30.252902 7513 raft_consensus.cc:2454] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ce24bd88bb940c7b2620b0662af02de in term 1.
I20260929 16:09:30.252909 7644 raft_consensus.cc:2454] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ce24bd88bb940c7b2620b0662af02de in term 1.
I20260929 16:09:30.253459 7709 leader_election.cc:302] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03462ddcfc5e456885ba66d1939621cc, 2ce24bd88bb940c7b2620b0662af02de, 3399676174d540a9a8e54656e02cd0ab; no voters:
I20260929 16:09:30.253851 7843 raft_consensus.cc:2781] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:30.254469 7843 raft_consensus.cc:686] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 1 LEADER]: Becoming Leader. State: Replica: 2ce24bd88bb940c7b2620b0662af02de, State: Running, Role: LEADER
I20260929 16:09:30.255002 7843 consensus_queue.cc:237] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 } } peers { permanent_uuid: "2ce24bd88bb940c7b2620b0662af02de" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45947 } } peers { permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 } }
I20260929 16:09:30.260248 7371 catalog_manager.cc:5495] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ce24bd88bb940c7b2620b0662af02de (127.20.121.131). New cstate: current_term: 1 leader_uuid: "2ce24bd88bb940c7b2620b0662af02de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ce24bd88bb940c7b2620b0662af02de" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 45947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 } health_report { overall_health: UNKNOWN } } }
I20260929 16:09:30.277702 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.271697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.20.121.131:45947)
0929 16:09:30.271872 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:30.271877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:30.271895 (+ 18us) server_negotiation.cc:407] Connection header received
0929 16:09:30.272086 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:30.272090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:30.272178 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:30.272304 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:09:30.272782 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.273614 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.275593 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.276009 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.276110 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:30.276585 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:09:30.277098 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:09:30.277226 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:30.277350 (+ 124us) server_negotiation.cc:299] Negotiation successful
0929 16:09:30.277427 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 16:09:30.286271 7644 raft_consensus.cc:1261] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc [term 1 FOLLOWER]: Refusing update from remote peer 2ce24bd88bb940c7b2620b0662af02de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:30.286273 7513 raft_consensus.cc:1261] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab [term 1 FOLLOWER]: Refusing update from remote peer 2ce24bd88bb940c7b2620b0662af02de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:30.287477 7843 consensus_queue.cc:1035] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [LEADER]: Connected to new peer: Peer: permanent_uuid: "3399676174d540a9a8e54656e02cd0ab" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 40259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:30.287806 7847 consensus_queue.cc:1035] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [LEADER]: Connected to new peer: Peer: permanent_uuid: "03462ddcfc5e456885ba66d1939621cc" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 41743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:30.299768 7850 mvcc.cc:205] Tried to move back new op lower bound from 7334699705481437184 to 7334699705368231936. Current Snapshot: MvccSnapshot[applied={T|T < 7334699705481437184}]
I20260929 16:09:30.304028 7851 mvcc.cc:205] Tried to move back new op lower bound from 7334699705481437184 to 7334699705368231936. Current Snapshot: MvccSnapshot[applied={T|T < 7334699705481437184}]
I20260929 16:09:30.316296 7371 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:30.316602 7371 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:30.320488 7371 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4c7ec1a6c24d48d38a79f99cd72082ba: Sending DeleteTablet for 3 replicas of tablet 84acb42475df4eb381e9e7578fe5cc1d
I20260929 16:09:30.321748 7493 tablet_service.cc:1514] Processing DeleteTablet for tablet 84acb42475df4eb381e9e7578fe5cc1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:30 UTC) from {username='slave'} at 127.0.0.1:34678
I20260929 16:09:30.321897 7754 tablet_service.cc:1514] Processing DeleteTablet for tablet 84acb42475df4eb381e9e7578fe5cc1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:30 UTC) from {username='slave'} at 127.0.0.1:49996
I20260929 16:09:30.323189 7624 tablet_service.cc:1514] Processing DeleteTablet for tablet 84acb42475df4eb381e9e7578fe5cc1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:30 UTC) from {username='slave'} at 127.0.0.1:56040
I20260929 16:09:30.324340 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 7431
I20260929 16:09:30.337335 7859 tablet_replica.cc:330] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: stopping tablet replica
I20260929 16:09:30.337857 7859 raft_consensus.cc:2229] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:09:30.338620 7859 raft_consensus.cc:2258] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:30.344223 7859 ts_tablet_manager.cc:1905] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:30.349256 7860 tablet_replica.cc:330] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab: stopping tablet replica
I20260929 16:09:30.349759 7860 raft_consensus.cc:2229] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:30.350354 7860 raft_consensus.cc:2258] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:30.355681 7861 tablet_replica.cc:330] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: stopping tablet replica
I20260929 16:09:30.355664 7860 ts_tablet_manager.cc:1905] T 84acb42475df4eb381e9e7578fe5cc1d P 3399676174d540a9a8e54656e02cd0ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:30.356273 7861 raft_consensus.cc:2229] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:30.356837 7861 raft_consensus.cc:2258] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:30.359481 7861 ts_tablet_manager.cc:1905] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:30.361514 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.325602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.20.121.129:40259)
0929 16:09:30.325835 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:30.325845 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:30.350912 (+ 25067us) server_negotiation.cc:407] Connection header received
0929 16:09:30.350998 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:30.351006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:30.351126 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:30.351232 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:30.352604 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.353752 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.357131 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.357803 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.357945 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:30.359582 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:30.359594 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:30.359613 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:30.360394 (+ 781us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:30.360455 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:30.360481 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:30.360570 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:30.360950 (+ 380us) server_negotiation.cc:299] Negotiation successful
0929 16:09:30.361087 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:09:30.363005 7553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:30.363487 7859 ts_tablet_manager.cc:1918] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:30.363680 7859 log.cc:1196] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/84acb42475df4eb381e9e7578fe5cc1d
I20260929 16:09:30.364279 7859 ts_tablet_manager.cc:1939] T 84acb42475df4eb381e9e7578fe5cc1d P 2ce24bd88bb940c7b2620b0662af02de: Deleting consensus metadata
I20260929 16:09:30.366236 7356 catalog_manager.cc:4841] TS 2ce24bd88bb940c7b2620b0662af02de (127.20.121.131:45947): tablet 84acb42475df4eb381e9e7578fe5cc1d (table test-table [id=bd0c3ae547284c78b3a789e498da031f]) successfully deleted
I20260929 16:09:30.366731 7861 ts_tablet_manager.cc:1918] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:30.366900 7861 log.cc:1196] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/84acb42475df4eb381e9e7578fe5cc1d
I20260929 16:09:30.367493 7861 ts_tablet_manager.cc:1939] T 84acb42475df4eb381e9e7578fe5cc1d P 03462ddcfc5e456885ba66d1939621cc: Deleting consensus metadata
I20260929 16:09:30.369091 7359 catalog_manager.cc:4841] TS 03462ddcfc5e456885ba66d1939621cc (127.20.121.130:41743): tablet 84acb42475df4eb381e9e7578fe5cc1d (table test-table [id=bd0c3ae547284c78b3a789e498da031f]) successfully deleted
I20260929 16:09:30.476684 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 7431
I20260929 16:09:30.495107 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 7562
W20260929 16:09:30.495139 7358 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.121.129:40259 (error 108)
I20260929 16:09:30.497439 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.496618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40259 (local address 127.0.0.1:34692)
0929 16:09:30.497074 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:30.497213 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:40259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":235,"threads_started":1}
W20260929 16:09:30.497745 7358 catalog_manager.cc:4584] TS 3399676174d540a9a8e54656e02cd0ab (127.20.121.129:40259): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 84acb42475df4eb381e9e7578fe5cc1d: Network error: Client connection negotiation failed: client connection to 127.20.121.129:40259: connect: Connection refused (error 111)
I20260929 16:09:30.503813 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.496324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.20.121.130:41743)
0929 16:09:30.496519 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:30.496527 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:30.496550 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:09:30.496618 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:30.496624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:30.496765 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:30.496882 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:30.497498 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.498703 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.500081 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.500737 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.500872 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:30.501973 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:30.501985 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:30.502004 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:30.502967 (+ 963us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:30.503029 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:30.503055 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:30.503173 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:30.503311 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 16:09:30.503427 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 16:09:30.504628 7684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:30.551440 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.550860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40259 (local address 127.0.0.1:34702)
0929 16:09:30.551128 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:30.551258 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:40259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260929 16:09:30.552701 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.552268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40259 (local address 127.0.0.1:34706)
0929 16:09:30.552454 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:30.552541 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:40259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 16:09:30.597361 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 7562
I20260929 16:09:30.613517 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.612997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40259 (local address 127.0.0.1:34716)
0929 16:09:30.613256 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:30.613349 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:40259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260929 16:09:30.614918 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.614496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40259 (local address 127.0.0.1:34722)
0929 16:09:30.614669 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:30.614770 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:40259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 16:09:30.621091 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 7692
I20260929 16:09:30.629607 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.621917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50020 (local address 127.20.121.131:45947)
0929 16:09:30.622140 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:30.622147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:30.622169 (+ 22us) server_negotiation.cc:407] Connection header received
0929 16:09:30.622319 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:30.622324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:30.622428 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:30.622556 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:30.622965 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.624129 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.625476 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:30.626234 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:30.626378 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:30.627643 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:30.627655 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:30.627675 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:30.628592 (+ 917us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:30.628659 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:30.628686 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:30.628810 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:30.629021 (+ 211us) server_negotiation.cc:299] Negotiation successful
0929 16:09:30.629158 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 16:09:30.630466 7814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:30.696096 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.695500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40259 (local address 127.0.0.1:34724)
0929 16:09:30.695790 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:30.695907 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:40259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260929 16:09:30.697454 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.696968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40259 (local address 127.0.0.1:34736)
0929 16:09:30.697194 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:30.697288 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:40259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260929 16:09:30.758184 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 7692
I20260929 16:09:30.778734 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 7341
I20260929 16:09:30.780158 7402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:30.893158 7865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:30.892420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:40259 (local address 127.0.0.1:34750)
0929 16:09:30.892823 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:30.892938 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.129:40259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198}
I20260929 16:09:30.893155 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 7341
2026-09-29T16:09:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3123 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:31 dist-test-slave-x07d krb5kdc[7871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:31 dist-test-slave-x07d krb5kdc[7871](info): set up 2 sockets
Sep 29 16:09:31 dist-test-slave-x07d krb5kdc[7871](info): commencing operation
krb5kdc: starting...
W20260929 16:09:32.023061 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:32 dist-test-slave-x07d krb5kdc[7871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:09:32.302294 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33617
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:33617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:32.649010 7884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:32.649336 7884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:32.649436 7884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:32.656785 7884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:32.656924 7884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:32.656972 7884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:32.657023 7884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:32.657071 7884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:32.667416 7884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:33617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:32.668937 7884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:32.670364 7884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:32.683924 7889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:32.683990 7890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:32.683954 7892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:32.684767 7884 server_base.cc:1034] running on GCE node
I20260929 16:09:32.685599 7884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:32.687175 7884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:32.688421 7884 hybrid_clock.cc:648] HybridClock initialized: now 1790698172688361 us; error 54 us; skew 500 ppm
I20260929 16:09:32.688839 7884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:32 dist-test-slave-x07d krb5kdc[7871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698172, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:32.692086 7884 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:09:32.693575 7884 webserver.cc:460] Webserver started at http://127.20.121.190:40539/ using document root <none> and password file <none>
I20260929 16:09:32.694324 7884 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:32.694424 7884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:32.694679 7884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:32.697692 7884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "b1917341e40d4ab2810156a7947dc880"
format_stamp: "Formatted at 2026-09-29 16:09:32 on dist-test-slave-x07d"
server_key: "b9c9d0788432cfd74f86cc6d1e7223df"
server_key_iv: "68488be4759921b3ab9486eacd802458"
server_key_version: "encryptionkey@0"
I20260929 16:09:32.698556 7884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "b1917341e40d4ab2810156a7947dc880"
format_stamp: "Formatted at 2026-09-29 16:09:32 on dist-test-slave-x07d"
server_key: "b9c9d0788432cfd74f86cc6d1e7223df"
server_key_iv: "68488be4759921b3ab9486eacd802458"
server_key_version: "encryptionkey@0"
I20260929 16:09:32.704371 7884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260929 16:09:32.708016 7899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:32.709542 7884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:09:32.709745 7884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "b1917341e40d4ab2810156a7947dc880"
format_stamp: "Formatted at 2026-09-29 16:09:32 on dist-test-slave-x07d"
server_key: "b9c9d0788432cfd74f86cc6d1e7223df"
server_key_iv: "68488be4759921b3ab9486eacd802458"
server_key_version: "encryptionkey@0"
I20260929 16:09:32.709975 7884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:32.725383 7884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:32.730659 7884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:32.730997 7884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:32.751396 7884 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:33617
I20260929 16:09:32.751415 7950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:33617 every 8 connection(s)
I20260929 16:09:32.752853 7884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:32.756772 7951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:32.759804 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 7884
I20260929 16:09:32.759994 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:32.760535 20966 external_mini_cluster.cc:1428] Setting key 93e3fa52ae18e5fd65ace647345809f5
I20260929 16:09:32.769011 7951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880: Bootstrap starting.
I20260929 16:09:32.773132 7951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:32.774628 7951 log.cc:824] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880: Log is configured to *not* fsync() on all Append() calls
Sep 29 16:09:32 dist-test-slave-x07d krb5kdc[7871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:32.778162 7951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880: No bootstrap required, opened a new log
I20260929 16:09:32.785725 7954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:32.762655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53714 (local address 127.20.121.190:33617)
0929 16:09:32.763518 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:32.763531 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:32.763572 (+ 41us) server_negotiation.cc:407] Connection header received
0929 16:09:32.765316 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:32.765342 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:32.765762 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:32.766255 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:32.767772 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:32.769095 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:32.771026 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:32.771790 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:32.771932 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:32.777337 (+ 5405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:32.777347 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:32.777363 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:32.777407 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:32.781211 (+ 3804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:32.782089 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:32.782097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:32.782104 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:32.782294 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:32.782728 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:32.782732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:32.782734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:32.783333 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:32.783663 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:32.784150 (+ 487us) server_negotiation.cc:299] Negotiation successful
0929 16:09:32.784612 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":379,"threads_started":1}
I20260929 16:09:32.786942 7951 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1917341e40d4ab2810156a7947dc880" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33617 } }
I20260929 16:09:32.787336 7951 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:32.787460 7951 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1917341e40d4ab2810156a7947dc880, State: Initialized, Role: FOLLOWER
I20260929 16:09:32.788129 7951 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1917341e40d4ab2810156a7947dc880" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33617 } }
I20260929 16:09:32.788372 7951 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:32.788525 7951 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:32.788719 7951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:32.791108 7951 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1917341e40d4ab2810156a7947dc880" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33617 } }
I20260929 16:09:32.791662 7951 leader_election.cc:302] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1917341e40d4ab2810156a7947dc880; no voters:
I20260929 16:09:32.792236 7951 leader_election.cc:288] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:32.792387 7956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:32.793797 7956 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [term 1 LEADER]: Becoming Leader. State: Replica: b1917341e40d4ab2810156a7947dc880, State: Running, Role: LEADER
I20260929 16:09:32.794432 7951 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:32.794350 7956 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1917341e40d4ab2810156a7947dc880" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33617 } }
I20260929 16:09:32.797451 7957 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1917341e40d4ab2810156a7947dc880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1917341e40d4ab2810156a7947dc880" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33617 } } }
I20260929 16:09:32.797851 7958 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1917341e40d4ab2810156a7947dc880. Latest consensus state: current_term: 1 leader_uuid: "b1917341e40d4ab2810156a7947dc880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1917341e40d4ab2810156a7947dc880" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33617 } } }
I20260929 16:09:32.798359 7958 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:32.798367 7957 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:32.801276 7966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:32.806069 7966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:32.815147 7966 catalog_manager.cc:1347] Generated new cluster ID: f6e6eb8d48104ad5ad49af88600ffd4e
I20260929 16:09:32.815290 7966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:32.830708 7966 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:32.832165 7966 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:32.847741 7966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880: Generated new TSK 0
I20260929 16:09:32.848685 7966 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.121.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
Entry for principal kudu/127.20.121.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab.
I20260929 16:09:32.925464 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33617
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:33.305032 7977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:33.305341 7977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:33.305438 7977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:33.314198 7977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:33.314358 7977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:33.314492 7977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:09:33.325572 7977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.129.keytab
--principal=kudu/127.20.121.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:33.327025 7977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:33.328369 7977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:33.342680 7982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:33.342656 7983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:33.342875 7985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:33.344269 7977 server_base.cc:1034] running on GCE node
I20260929 16:09:33.345027 7977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:33.346203 7977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:33.347478 7977 hybrid_clock.cc:648] HybridClock initialized: now 1790698173347429 us; error 43 us; skew 500 ppm
I20260929 16:09:33.347886 7977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:33 dist-test-slave-x07d krb5kdc[7871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698173, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:33.351467 7977 init.cc:377] Logged in from keytab as kudu/127.20.121.129@KRBTEST.COM (short username kudu)
I20260929 16:09:33.353077 7977 webserver.cc:460] Webserver started at http://127.20.121.129:36501/ using document root <none> and password file <none>
I20260929 16:09:33.353797 7977 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:33.353907 7977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:33.354265 7977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:33.357357 7977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "d994660c32bf4e29a64d6ea89780da47"
format_stamp: "Formatted at 2026-09-29 16:09:33 on dist-test-slave-x07d"
server_key: "be00ee891ad6331640ca2eae48a212d6"
server_key_iv: "b8e43c696abdfd55f88dca42d590b946"
server_key_version: "encryptionkey@0"
I20260929 16:09:33.358208 7977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "d994660c32bf4e29a64d6ea89780da47"
format_stamp: "Formatted at 2026-09-29 16:09:33 on dist-test-slave-x07d"
server_key: "be00ee891ad6331640ca2eae48a212d6"
server_key_iv: "b8e43c696abdfd55f88dca42d590b946"
server_key_version: "encryptionkey@0"
I20260929 16:09:33.364235 7977 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260929 16:09:33.368512 7992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:33.370249 7977 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:09:33.370405 7977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "d994660c32bf4e29a64d6ea89780da47"
format_stamp: "Formatted at 2026-09-29 16:09:33 on dist-test-slave-x07d"
server_key: "be00ee891ad6331640ca2eae48a212d6"
server_key_iv: "b8e43c696abdfd55f88dca42d590b946"
server_key_version: "encryptionkey@0"
I20260929 16:09:33.370596 7977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:33.397495 7977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:33.402328 7977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:33.402603 7977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:33.403616 7977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:33.404991 7977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:33.405068 7977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:33.405157 7977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:33.405201 7977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:33.440898 7977 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:35023
I20260929 16:09:33.441087 8104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:35023 every 8 connection(s)
I20260929 16:09:33.442327 7977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:09:33.449455 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 7977
I20260929 16:09:33.449667 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:09:33.450155 20966 external_mini_cluster.cc:1428] Setting key 942ac4a330fc193c6ae00484628838fc
Sep 29 16:09:33 dist-test-slave-x07d krb5kdc[7871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698173, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.129@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:33.466125 8107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:33.445064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:43219 (local address 127.20.121.190:33617)
0929 16:09:33.445574 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:33.445579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:33.446577 (+ 998us) server_negotiation.cc:407] Connection header received
0929 16:09:33.448827 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:33.448834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:33.448937 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:33.449089 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:33.451002 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:33.451973 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:33.453661 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:33.454416 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:33.454569 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:33.458960 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:33.458968 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:33.458976 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:33.459026 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:33.462277 (+ 3251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:33.463389 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:33.463393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:33.463396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:33.463529 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:33.464111 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:33.464115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:33.464117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:33.464419 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:33.464542 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:33.465564 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0929 16:09:33.465762 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":304,"threads_started":1}
I20260929 16:09:33.467626 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:33.445574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33617 (local address 127.20.121.129:43219)
0929 16:09:33.446401 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:33.446441 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:33.448497 (+ 2056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:33.449325 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:33.449335 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:33.450004 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:33.450795 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:33.450824 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:33.452161 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:33.452167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:33.453494 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:33.453508 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:33.454752 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:33.454758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:33.455028 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:33.455876 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:09:33.455902 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:33.458627 (+ 2725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:33.462449 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:33.462464 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:33.462487 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:33.463210 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:33.463655 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:33.463663 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:33.463669 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:33.463966 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:33.464568 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:33.464579 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:33.465122 (+ 543us) client_negotiation.cc:769] Sending connection context
0929 16:09:33.465442 (+ 320us) client_negotiation.cc:240] Negotiation successful
0929 16:09:33.466103 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":295,"threads_started":1}
I20260929 16:09:33.469558 8105 heartbeater.cc:347] Connected to a master server at 127.20.121.190:33617
I20260929 16:09:33.469935 8105 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:33.470968 8105 heartbeater.cc:511] Master 127.20.121.190:33617 requested a full tablet report, sending...
I20260929 16:09:33.473796 7916 ts_manager.cc:194] Registered new tserver with Master: d994660c32bf4e29a64d6ea89780da47 (127.20.121.129:35023)
I20260929 16:09:33.476261 7916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.129@KRBTEST.COM'} at 127.20.121.129:43219
WARNING: no policy specified for kudu/127.20.121.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
Entry for principal kudu/127.20.121.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab.
I20260929 16:09:33.513839 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33617
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:33.858414 8111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:33.858808 8111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:33.858968 8111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:33.866495 8111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:33.866693 8111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:33.866891 8111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:09:33.877660 8111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.130.keytab
--principal=kudu/127.20.121.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:33.879117 8111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:33.880554 8111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:33.894979 8117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:33.895033 8116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:33.895207 8119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:33.896234 8111 server_base.cc:1034] running on GCE node
I20260929 16:09:33.896955 8111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:33.898211 8111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:33.899482 8111 hybrid_clock.cc:648] HybridClock initialized: now 1790698173899406 us; error 60 us; skew 500 ppm
I20260929 16:09:33.899866 8111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:33 dist-test-slave-x07d krb5kdc[7871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698173, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:33.903234 8111 init.cc:377] Logged in from keytab as kudu/127.20.121.130@KRBTEST.COM (short username kudu)
I20260929 16:09:33.904793 8111 webserver.cc:460] Webserver started at http://127.20.121.130:39657/ using document root <none> and password file <none>
I20260929 16:09:33.905483 8111 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:33.905586 8111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:33.905871 8111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:33.909098 8111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "6006466a50004d17a7867e6d8af2e4c6"
format_stamp: "Formatted at 2026-09-29 16:09:33 on dist-test-slave-x07d"
server_key: "e61225122b3fa216c106ef8b7f9ab7b4"
server_key_iv: "37ef848694f1897215229444f486100f"
server_key_version: "encryptionkey@0"
I20260929 16:09:33.909977 8111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "6006466a50004d17a7867e6d8af2e4c6"
format_stamp: "Formatted at 2026-09-29 16:09:33 on dist-test-slave-x07d"
server_key: "e61225122b3fa216c106ef8b7f9ab7b4"
server_key_iv: "37ef848694f1897215229444f486100f"
server_key_version: "encryptionkey@0"
I20260929 16:09:33.916002 8111 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260929 16:09:33.919947 8126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:33.921474 8111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:09:33.921665 8111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "6006466a50004d17a7867e6d8af2e4c6"
format_stamp: "Formatted at 2026-09-29 16:09:33 on dist-test-slave-x07d"
server_key: "e61225122b3fa216c106ef8b7f9ab7b4"
server_key_iv: "37ef848694f1897215229444f486100f"
server_key_version: "encryptionkey@0"
I20260929 16:09:33.921877 8111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:33.956000 8111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:33.960855 8111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:33.961191 8111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:33.962332 8111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:33.963757 8111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:33.963831 8111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:33.964018 8111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:33.964071 8111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:34.000264 8111 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:37977
I20260929 16:09:34.000394 8238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:37977 every 8 connection(s)
I20260929 16:09:34.001652 8111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:09:34.010900 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 8111
I20260929 16:09:34.011152 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:09:34.011650 20966 external_mini_cluster.cc:1428] Setting key cc380f380115883ceb2cc5a155b09d9e
Sep 29 16:09:34 dist-test-slave-x07d krb5kdc[7871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698173, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.130@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:34.027988 8241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.004689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:51481 (local address 127.20.121.190:33617)
0929 16:09:34.005196 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:34.005203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:34.006473 (+ 1270us) server_negotiation.cc:407] Connection header received
0929 16:09:34.008554 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:34.008558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:34.008639 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:34.008761 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:34.011047 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.011923 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.014503 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.015116 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.015226 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.020580 (+ 5354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:34.020587 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:34.020591 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:34.020633 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:34.023917 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:34.024885 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:34.024891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:34.024897 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:34.025041 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:34.025533 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:34.025537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:34.025539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:34.025940 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:34.026226 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:34.027047 (+ 821us) server_negotiation.cc:299] Negotiation successful
0929 16:09:34.027370 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":255,"threads_started":1}
I20260929 16:09:34.028534 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.005393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33617 (local address 127.20.121.130:51481)
0929 16:09:34.006273 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:34.006329 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:34.008213 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:34.009104 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:34.009114 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:34.009741 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:34.010711 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.010753 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.012105 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.012113 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.014303 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.014321 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.015416 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.015428 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.015857 (+ 429us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.016801 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:09:34.016848 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:34.020082 (+ 3234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:34.024142 (+ 4060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:34.024152 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:34.024172 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:34.024723 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:34.025170 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:34.025176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:34.025179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:34.025390 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:34.026102 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:34.026110 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:34.026565 (+ 455us) client_negotiation.cc:769] Sending connection context
0929 16:09:34.026914 (+ 349us) client_negotiation.cc:240] Negotiation successful
0929 16:09:34.027363 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":317,"threads_started":1}
I20260929 16:09:34.030331 8239 heartbeater.cc:347] Connected to a master server at 127.20.121.190:33617
I20260929 16:09:34.030699 8239 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:34.031697 8239 heartbeater.cc:511] Master 127.20.121.190:33617 requested a full tablet report, sending...
I20260929 16:09:34.033792 7916 ts_manager.cc:194] Registered new tserver with Master: 6006466a50004d17a7867e6d8af2e4c6 (127.20.121.130:37977)
I20260929 16:09:34.034749 7916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.130@KRBTEST.COM'} at 127.20.121.130:51481
WARNING: no policy specified for kudu/127.20.121.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
Entry for principal kudu/127.20.121.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab.
I20260929 16:09:34.077425 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33617
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:34.425462 8245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:34.425899 8245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:34.426064 8245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:34.433697 8245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:34.433822 8245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:34.433939 8245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:09:34.447096 8245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.131.keytab
--principal=kudu/127.20.121.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:33617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:34.448469 8245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:34.449796 8245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:34.464470 8251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:34.464540 8250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:34.464514 8253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:34.465615 8245 server_base.cc:1034] running on GCE node
I20260929 16:09:34.466440 8245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:34.467586 8245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:34.468870 8245 hybrid_clock.cc:648] HybridClock initialized: now 1790698174468794 us; error 43 us; skew 500 ppm
I20260929 16:09:34.469234 8245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:34 dist-test-slave-x07d krb5kdc[7871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698174, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:34.472771 8245 init.cc:377] Logged in from keytab as kudu/127.20.121.131@KRBTEST.COM (short username kudu)
I20260929 16:09:34.474426 8245 webserver.cc:460] Webserver started at http://127.20.121.131:43093/ using document root <none> and password file <none>
I20260929 16:09:34.475186 8245 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:34.475292 8245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:34.475623 8245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:34.478873 8245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "e0c4a753d2144f7c9f24868056279187"
format_stamp: "Formatted at 2026-09-29 16:09:34 on dist-test-slave-x07d"
server_key: "d80ba7d165d62248d4aff071d7b28e19"
server_key_iv: "4e3faedbac433de40a2ec7ad0a12be94"
server_key_version: "encryptionkey@0"
I20260929 16:09:34.479698 8245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "e0c4a753d2144f7c9f24868056279187"
format_stamp: "Formatted at 2026-09-29 16:09:34 on dist-test-slave-x07d"
server_key: "d80ba7d165d62248d4aff071d7b28e19"
server_key_iv: "4e3faedbac433de40a2ec7ad0a12be94"
server_key_version: "encryptionkey@0"
I20260929 16:09:34.480446 8105 heartbeater.cc:503] Master 127.20.121.190:33617 was elected leader, sending a full tablet report...
I20260929 16:09:34.485846 8245 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260929 16:09:34.490284 8260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:34.492434 8245 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 16:09:34.492682 8245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "e0c4a753d2144f7c9f24868056279187"
format_stamp: "Formatted at 2026-09-29 16:09:34 on dist-test-slave-x07d"
server_key: "d80ba7d165d62248d4aff071d7b28e19"
server_key_iv: "4e3faedbac433de40a2ec7ad0a12be94"
server_key_version: "encryptionkey@0"
I20260929 16:09:34.492935 8245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:34.530966 8245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:34.536499 8245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:34.536954 8245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:34.538236 8245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:34.540012 8245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:34.540096 8245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:34.540244 8245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:34.540310 8245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:34.577703 8245 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:38539
I20260929 16:09:34.577739 8372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:38539 every 8 connection(s)
I20260929 16:09:34.579311 8245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:09:34.584884 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 8245
I20260929 16:09:34.585105 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:09:34.585763 20966 external_mini_cluster.cc:1428] Setting key f2218dfb4ffc0862fe85da5bfd98a433
Sep 29 16:09:34 dist-test-slave-x07d krb5kdc[7871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698174, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.131@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:09:34.604632 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.582490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:50503 (local address 127.20.121.190:33617)
0929 16:09:34.582991 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:34.582996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:34.584194 (+ 1198us) server_negotiation.cc:407] Connection header received
0929 16:09:34.587132 (+ 2938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:34.587138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:34.587271 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:34.587421 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:34.589223 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.590210 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.592024 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.592491 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.592649 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.597099 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:34.597105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:34.597109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:34.597162 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:34.600889 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:34.601920 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:34.601924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:34.601927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:34.602171 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:34.602747 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:34.602751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:34.602753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:34.603094 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:34.603219 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:34.604118 (+ 899us) server_negotiation.cc:299] Negotiation successful
0929 16:09:34.604331 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":246,"threads_started":1}
I20260929 16:09:34.605770 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.583162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33617 (local address 127.20.121.131:50503)
0929 16:09:34.583985 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:34.584036 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:34.586774 (+ 2738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:34.587723 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:34.587734 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:34.588347 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:34.589024 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.589044 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.590399 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.590404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.591841 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.591856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.592764 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.592775 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.593064 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.593885 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:09:34.593911 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:34.596765 (+ 2854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:34.601093 (+ 4328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:34.601104 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:34.601126 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:34.601710 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:34.602353 (+ 643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:09:34.602359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:09:34.602363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:09:34.602627 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:09:34.603233 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:34.603241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:34.603650 (+ 409us) client_negotiation.cc:769] Sending connection context
0929 16:09:34.604007 (+ 357us) client_negotiation.cc:240] Negotiation successful
0929 16:09:34.604470 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":337,"threads_started":1}
I20260929 16:09:34.608026 8373 heartbeater.cc:347] Connected to a master server at 127.20.121.190:33617
I20260929 16:09:34.608350 8373 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:34.609134 8373 heartbeater.cc:511] Master 127.20.121.190:33617 requested a full tablet report, sending...
I20260929 16:09:34.610993 7916 ts_manager.cc:194] Registered new tserver with Master: e0c4a753d2144f7c9f24868056279187 (127.20.121.131:38539)
I20260929 16:09:34.611816 7916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.121.131@KRBTEST.COM'} at 127.20.121.131:50503
I20260929 16:09:34.614642 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:09:34.633658 8375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.619171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53720 (local address 127.20.121.190:33617)
0929 16:09:34.619365 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:34.619370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:34.619720 (+ 350us) server_negotiation.cc:407] Connection header received
0929 16:09:34.619880 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:34.619884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:34.619975 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:34.620077 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:34.621617 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.622500 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.623992 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.624479 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.624591 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.627753 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:34.627758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:34.627762 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:34.627804 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:34.630852 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:34.631605 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:34.631610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:34.631613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:34.631746 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:34.632159 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:34.632162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:34.632165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:34.632525 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:34.632726 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:34.633180 (+ 454us) server_negotiation.cc:299] Negotiation successful
0929 16:09:34.633401 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 16:09:34.638486 7916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:09:34.646543 7916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:09:34.684984 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.664132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.20.121.131:38539)
0929 16:09:34.673149 (+ 9017us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:34.673164 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:34.675355 (+ 2191us) server_negotiation.cc:407] Connection header received
0929 16:09:34.675696 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:34.675705 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:34.675965 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:34.676166 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:34.677558 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.679294 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.682449 (+ 3155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.683572 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.683823 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.684047 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:34.684347 (+ 300us) server_negotiation.cc:299] Negotiation successful
0929 16:09:34.684533 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8726,"thread_start_us":8454,"threads_started":1}
I20260929 16:09:34.685086 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.663217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:38539 (local address 127.0.0.1:45808)
0929 16:09:34.675220 (+ 12003us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:34.675240 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:34.675479 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:34.676520 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:34.676528 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:34.676561 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:34.677282 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.677304 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.679466 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.679472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.682252 (+ 2780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.682268 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.683819 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.683826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.684054 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.684130 (+ 76us) client_negotiation.cc:769] Sending connection context
0929 16:09:34.684287 (+ 157us) client_negotiation.cc:240] Negotiation successful
0929 16:09:34.684540 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11866,"mutex_wait_us":209,"thread_start_us":7824,"threads_started":1}
I20260929 16:09:34.685856 8388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.661464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35023 (local address 127.0.0.1:44292)
0929 16:09:34.673835 (+ 12371us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:34.673859 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:34.674035 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:34.676624 (+ 2589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:34.676630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:34.676672 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:34.677384 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.677398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.679272 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.679278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.682250 (+ 2972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.682270 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.684765 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.684772 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.685079 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.685113 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 16:09:34.685210 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 16:09:34.685362 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12218,"mutex_wait_us":185,"thread_start_us":12045,"threads_started":1}
I20260929 16:09:34.686494 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.661361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:37977 (local address 127.0.0.1:38610)
0929 16:09:34.671703 (+ 10342us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:34.671785 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:34.672136 (+ 351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:34.675683 (+ 3547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:34.675691 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:34.675731 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:34.676200 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.676219 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.678813 (+ 2594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.678819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.682066 (+ 3247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.682093 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.684746 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.684754 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.685736 (+ 982us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.685782 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 16:09:34.685910 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 16:09:34.686106 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9904,"mutex_wait_us":745,"thread_start_us":13556,"threads_started":1}
I20260929 16:09:34.686820 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.662301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44292 (local address 127.20.121.129:35023)
0929 16:09:34.675747 (+ 13446us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:34.675754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:34.675780 (+ 26us) server_negotiation.cc:407] Connection header received
0929 16:09:34.675874 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:34.675880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:34.676056 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:34.676321 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:34.677611 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.679097 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.682502 (+ 3405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.684157 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.684404 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.684568 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:34.686234 (+ 1666us) server_negotiation.cc:299] Negotiation successful
0929 16:09:34.686453 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13263,"thread_start_us":13092,"threads_started":1}
I20260929 16:09:34.687153 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.661470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38610 (local address 127.20.121.130:37977)
0929 16:09:34.674761 (+ 13291us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:34.674772 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:34.674893 (+ 121us) server_negotiation.cc:407] Connection header received
0929 16:09:34.675047 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:34.675056 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:34.675387 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:34.676436 (+ 1049us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:34.676752 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.678639 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.682303 (+ 3664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.684157 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.684347 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.684523 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:34.686644 (+ 2121us) server_negotiation.cc:299] Negotiation successful
0929 16:09:34.686840 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13141,"thread_start_us":13028,"threads_started":1}
I20260929 16:09:34.690339 8174 tablet_service.cc:1467] Processing CreateTablet for tablet 384b31eee12a4669a4ec92c9ac9f8b3b (DEFAULT_TABLE table=test-table [id=1ccd433cff864676bd7883729872e450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:34.690126 8308 tablet_service.cc:1467] Processing CreateTablet for tablet 384b31eee12a4669a4ec92c9ac9f8b3b (DEFAULT_TABLE table=test-table [id=1ccd433cff864676bd7883729872e450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:34.690950 8040 tablet_service.cc:1467] Processing CreateTablet for tablet 384b31eee12a4669a4ec92c9ac9f8b3b (DEFAULT_TABLE table=test-table [id=1ccd433cff864676bd7883729872e450]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:34.692044 8174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384b31eee12a4669a4ec92c9ac9f8b3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:34.692127 8308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384b31eee12a4669a4ec92c9ac9f8b3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:34.692337 8040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384b31eee12a4669a4ec92c9ac9f8b3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:34.707139 8392 tablet_bootstrap.cc:492] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: Bootstrap starting.
I20260929 16:09:34.709455 8393 tablet_bootstrap.cc:492] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: Bootstrap starting.
I20260929 16:09:34.710419 8394 tablet_bootstrap.cc:492] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: Bootstrap starting.
I20260929 16:09:34.711843 8392 tablet_bootstrap.cc:654] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:34.713407 8392 log.cc:824] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:34.714385 8393 tablet_bootstrap.cc:654] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:34.714591 8394 tablet_bootstrap.cc:654] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:34.716065 8393 log.cc:824] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:34.716362 8394 log.cc:824] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:34.719982 8393 tablet_bootstrap.cc:492] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: No bootstrap required, opened a new log
I20260929 16:09:34.720335 8393 ts_tablet_manager.cc:1397] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260929 16:09:34.720661 8392 tablet_bootstrap.cc:492] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: No bootstrap required, opened a new log
I20260929 16:09:34.720957 8392 ts_tablet_manager.cc:1397] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260929 16:09:34.724530 8394 tablet_bootstrap.cc:492] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: No bootstrap required, opened a new log
I20260929 16:09:34.724864 8394 ts_tablet_manager.cc:1397] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260929 16:09:34.729560 8393 raft_consensus.cc:348] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 } } peers { permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 } } peers { permanent_uuid: "e0c4a753d2144f7c9f24868056279187" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 38539 } }
I20260929 16:09:34.730096 8393 raft_consensus.cc:374] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:34.730264 8393 raft_consensus.cc:729] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d994660c32bf4e29a64d6ea89780da47, State: Initialized, Role: FOLLOWER
I20260929 16:09:34.731014 8393 consensus_queue.cc:260] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 } } peers { permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 } } peers { permanent_uuid: "e0c4a753d2144f7c9f24868056279187" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 38539 } }
I20260929 16:09:34.733178 8392 raft_consensus.cc:348] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 } } peers { permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 } } peers { permanent_uuid: "e0c4a753d2144f7c9f24868056279187" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 38539 } }
I20260929 16:09:34.733796 8392 raft_consensus.cc:374] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:34.733969 8392 raft_consensus.cc:729] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6006466a50004d17a7867e6d8af2e4c6, State: Initialized, Role: FOLLOWER
I20260929 16:09:34.734256 8393 ts_tablet_manager.cc:1428] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260929 16:09:34.734887 8392 consensus_queue.cc:260] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 } } peers { permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 } } peers { permanent_uuid: "e0c4a753d2144f7c9f24868056279187" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 38539 } }
I20260929 16:09:34.736333 8239 heartbeater.cc:503] Master 127.20.121.190:33617 was elected leader, sending a full tablet report...
I20260929 16:09:34.737068 8392 ts_tablet_manager.cc:1428] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
I20260929 16:09:34.737429 8394 raft_consensus.cc:348] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 } } peers { permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 } } peers { permanent_uuid: "e0c4a753d2144f7c9f24868056279187" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 38539 } }
I20260929 16:09:34.737906 8394 raft_consensus.cc:374] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:34.738080 8394 raft_consensus.cc:729] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c4a753d2144f7c9f24868056279187, State: Initialized, Role: FOLLOWER
I20260929 16:09:34.738854 8394 consensus_queue.cc:260] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 } } peers { permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 } } peers { permanent_uuid: "e0c4a753d2144f7c9f24868056279187" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 38539 } }
I20260929 16:09:34.741660 8373 heartbeater.cc:503] Master 127.20.121.190:33617 was elected leader, sending a full tablet report...
I20260929 16:09:34.742650 8394 ts_tablet_manager.cc:1428] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: Time spent starting tablet: real 0.018s user 0.020s sys 0.000s
W20260929 16:09:34.754122 8240 tablet.cc:2369] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 16:09:34.831526 8374 tablet.cc:2369] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:09:34.940857 8400 raft_consensus.cc:482] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:34.941218 8400 raft_consensus.cc:504] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 } } peers { permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 } } peers { permanent_uuid: "e0c4a753d2144f7c9f24868056279187" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 38539 } }
I20260929 16:09:34.943051 8400 leader_election.cc:288] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6006466a50004d17a7867e6d8af2e4c6 (127.20.121.130:37977), d994660c32bf4e29a64d6ea89780da47 (127.20.121.129:35023)
W20260929 16:09:34.945215 8106 tablet.cc:2369] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:09:34.949158 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.943599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:39999 (local address 127.20.121.129:35023)
0929 16:09:34.943804 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:34.943810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:34.943834 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:09:34.943944 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:34.943951 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:34.944041 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:34.944163 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:34.944644 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.945766 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.947420 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.948225 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.948410 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.948486 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:34.948765 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 16:09:34.948907 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:09:34.949158 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.943422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:35023 (local address 127.20.121.131:39999)
0929 16:09:34.943612 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:34.943636 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:34.943788 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:34.944233 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:34.944237 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:34.944260 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:34.944473 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.944482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.945897 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.945905 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.947246 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.947260 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.948408 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.948414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.948600 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.948625 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 16:09:34.948736 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 16:09:34.948881 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 16:09:34.950430 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.943619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:53441 (local address 127.20.121.130:37977)
0929 16:09:34.943816 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:34.943824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:34.944453 (+ 629us) server_negotiation.cc:407] Connection header received
0929 16:09:34.944703 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:34.944707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:34.944774 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:34.944943 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:34.945595 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.946633 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.948512 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.949523 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.949735 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.949814 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:34.950111 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 16:09:34.950219 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 16:09:34.950615 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.943422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:37977 (local address 127.20.121.131:53441)
0929 16:09:34.944328 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:34.944347 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:34.944505 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:34.944955 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:34.944962 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:34.944994 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:34.945351 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.945363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.946808 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.946814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.948307 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:34.948320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.949753 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.949758 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:34.949881 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.949913 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 16:09:34.950079 (+ 166us) client_negotiation.cc:240] Negotiation successful
0929 16:09:34.950226 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":217,"thread_start_us":324,"threads_started":1}
I20260929 16:09:34.950544 8060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384b31eee12a4669a4ec92c9ac9f8b3b" candidate_uuid: "e0c4a753d2144f7c9f24868056279187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d994660c32bf4e29a64d6ea89780da47" is_pre_election: true
I20260929 16:09:34.951203 8060 raft_consensus.cc:2454] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0c4a753d2144f7c9f24868056279187 in term 0.
I20260929 16:09:34.952090 8264 leader_election.cc:302] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d994660c32bf4e29a64d6ea89780da47, e0c4a753d2144f7c9f24868056279187; no voters:
I20260929 16:09:34.951992 8194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384b31eee12a4669a4ec92c9ac9f8b3b" candidate_uuid: "e0c4a753d2144f7c9f24868056279187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6006466a50004d17a7867e6d8af2e4c6" is_pre_election: true
I20260929 16:09:34.952582 8400 raft_consensus.cc:2781] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:09:34.952644 8194 raft_consensus.cc:2454] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0c4a753d2144f7c9f24868056279187 in term 0.
I20260929 16:09:34.952780 8400 raft_consensus.cc:482] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:09:34.952903 8400 raft_consensus.cc:3037] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:34.955533 8400 raft_consensus.cc:504] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 } } peers { permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 } } peers { permanent_uuid: "e0c4a753d2144f7c9f24868056279187" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 38539 } }
I20260929 16:09:34.956166 8400 leader_election.cc:288] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [CANDIDATE]: Term 1 election: Requested vote from peers 6006466a50004d17a7867e6d8af2e4c6 (127.20.121.130:37977), d994660c32bf4e29a64d6ea89780da47 (127.20.121.129:35023)
I20260929 16:09:34.956804 8194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384b31eee12a4669a4ec92c9ac9f8b3b" candidate_uuid: "e0c4a753d2144f7c9f24868056279187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6006466a50004d17a7867e6d8af2e4c6"
I20260929 16:09:34.956804 8060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384b31eee12a4669a4ec92c9ac9f8b3b" candidate_uuid: "e0c4a753d2144f7c9f24868056279187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d994660c32bf4e29a64d6ea89780da47"
I20260929 16:09:34.957057 8060 raft_consensus.cc:3037] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:34.957057 8194 raft_consensus.cc:3037] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:34.960738 8194 raft_consensus.cc:2454] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0c4a753d2144f7c9f24868056279187 in term 1.
I20260929 16:09:34.960738 8060 raft_consensus.cc:2454] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0c4a753d2144f7c9f24868056279187 in term 1.
I20260929 16:09:34.961314 8264 leader_election.cc:302] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6006466a50004d17a7867e6d8af2e4c6, d994660c32bf4e29a64d6ea89780da47, e0c4a753d2144f7c9f24868056279187; no voters:
I20260929 16:09:34.961689 8400 raft_consensus.cc:2781] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:34.962304 8400 raft_consensus.cc:686] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 1 LEADER]: Becoming Leader. State: Replica: e0c4a753d2144f7c9f24868056279187, State: Running, Role: LEADER
I20260929 16:09:34.962832 8400 consensus_queue.cc:237] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 } } peers { permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 } } peers { permanent_uuid: "e0c4a753d2144f7c9f24868056279187" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 38539 } }
I20260929 16:09:34.967935 7916 catalog_manager.cc:5495] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0c4a753d2144f7c9f24868056279187 (127.20.121.131). New cstate: current_term: 1 leader_uuid: "e0c4a753d2144f7c9f24868056279187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0c4a753d2144f7c9f24868056279187" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 38539 } health_report { overall_health: HEALTHY } } }
I20260929 16:09:34.985662 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:34.979987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45816 (local address 127.20.121.131:38539)
0929 16:09:34.980182 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:34.980188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:34.980350 (+ 162us) server_negotiation.cc:407] Connection header received
0929 16:09:34.980491 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:34.980495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:34.980575 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:34.980721 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:09:34.981146 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.981907 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.983481 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:34.983898 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:34.983997 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:34.984553 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:09:34.985096 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:09:34.985216 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:34.985382 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 16:09:34.985454 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 16:09:34.995307 8060 raft_consensus.cc:1261] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47 [term 1 FOLLOWER]: Refusing update from remote peer e0c4a753d2144f7c9f24868056279187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:34.995308 8194 raft_consensus.cc:1261] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6 [term 1 FOLLOWER]: Refusing update from remote peer e0c4a753d2144f7c9f24868056279187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:34.996502 8402 consensus_queue.cc:1035] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6006466a50004d17a7867e6d8af2e4c6" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:34.997010 8400 consensus_queue.cc:1035] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d994660c32bf4e29a64d6ea89780da47" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 35023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:35.014700 8405 mvcc.cc:205] Tried to move back new op lower bound from 7334699724768223232 to 7334699724651773952. Current Snapshot: MvccSnapshot[applied={T|T < 7334699724768223232}]
I20260929 16:09:35.036020 7916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:35.036401 7916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:35.041105 7916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1917341e40d4ab2810156a7947dc880: Sending DeleteTablet for 3 replicas of tablet 384b31eee12a4669a4ec92c9ac9f8b3b
I20260929 16:09:35.042586 8308 tablet_service.cc:1514] Processing DeleteTablet for tablet 384b31eee12a4669a4ec92c9ac9f8b3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:35 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:45808
I20260929 16:09:35.042578 8174 tablet_service.cc:1514] Processing DeleteTablet for tablet 384b31eee12a4669a4ec92c9ac9f8b3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:35 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:38610
I20260929 16:09:35.043316 8040 tablet_service.cc:1514] Processing DeleteTablet for tablet 384b31eee12a4669a4ec92c9ac9f8b3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:35 UTC) from {username='kudu', principal='kudu/127.20.121.190@KRBTEST.COM'} at 127.0.0.1:44292
I20260929 16:09:35.045616 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 7977
I20260929 16:09:35.072543 8417 tablet_replica.cc:330] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: stopping tablet replica
I20260929 16:09:35.073385 8417 raft_consensus.cc:2229] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:35.076638 8416 tablet_replica.cc:330] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: stopping tablet replica
I20260929 16:09:35.077387 8415 tablet_replica.cc:330] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: stopping tablet replica
I20260929 16:09:35.078094 8415 raft_consensus.cc:2229] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:09:35.078630 8416 raft_consensus.cc:2229] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:35.079180 8417 raft_consensus.cc:2258] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:35.079255 8416 raft_consensus.cc:2258] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:35.080338 8415 raft_consensus.cc:2258] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:35.082504 8417 ts_tablet_manager.cc:1905] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:35.082604 8415 ts_tablet_manager.cc:1905] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:35.082932 8416 ts_tablet_manager.cc:1905] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:35.090643 8417 ts_tablet_manager.cc:1918] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:35.090644 8415 ts_tablet_manager.cc:1918] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:35.090863 8415 log.cc:1196] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/384b31eee12a4669a4ec92c9ac9f8b3b
I20260929 16:09:35.090989 8417 log.cc:1196] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/384b31eee12a4669a4ec92c9ac9f8b3b
I20260929 16:09:35.091668 8415 ts_tablet_manager.cc:1939] T 384b31eee12a4669a4ec92c9ac9f8b3b P e0c4a753d2144f7c9f24868056279187: Deleting consensus metadata
I20260929 16:09:35.091761 8417 ts_tablet_manager.cc:1939] T 384b31eee12a4669a4ec92c9ac9f8b3b P d994660c32bf4e29a64d6ea89780da47: Deleting consensus metadata
Sep 29 16:09:35 dist-test-slave-x07d krb5kdc[7871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.129@KRBTEST.COM
I20260929 16:09:35.093946 7903 catalog_manager.cc:4841] TS d994660c32bf4e29a64d6ea89780da47 (127.20.121.129:35023): tablet 384b31eee12a4669a4ec92c9ac9f8b3b (table test-table [id=1ccd433cff864676bd7883729872e450]) successfully deleted
I20260929 16:09:35.094374 7902 catalog_manager.cc:4841] TS e0c4a753d2144f7c9f24868056279187 (127.20.121.131:38539): tablet 384b31eee12a4669a4ec92c9ac9f8b3b (table test-table [id=1ccd433cff864676bd7883729872e450]) successfully deleted
I20260929 16:09:35.098896 8416 ts_tablet_manager.cc:1918] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:35.099139 8416 log.cc:1196] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/384b31eee12a4669a4ec92c9ac9f8b3b
I20260929 16:09:35.099853 8416 ts_tablet_manager.cc:1939] T 384b31eee12a4669a4ec92c9ac9f8b3b P 6006466a50004d17a7867e6d8af2e4c6: Deleting consensus metadata
I20260929 16:09:35.102265 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:35.047530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44306 (local address 127.20.121.129:35023)
0929 16:09:35.047798 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:35.047807 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:35.068317 (+ 20510us) server_negotiation.cc:407] Connection header received
0929 16:09:35.068996 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:35.069006 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:35.069147 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:35.069251 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:35.073878 (+ 4627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:35.078848 (+ 4970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:35.083197 (+ 4349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:35.083931 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:35.084068 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:35.094253 (+ 10185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:35.094269 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:35.094292 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:35.094352 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:35.098276 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:35.099546 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:35.099564 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:35.099567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:35.099726 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:35.100245 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:35.100251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:35.100254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:35.100670 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:35.101034 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:35.101373 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 16:09:35.101625 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260929 16:09:35.102308 7901 catalog_manager.cc:4841] TS 6006466a50004d17a7867e6d8af2e4c6 (127.20.121.130:37977): tablet 384b31eee12a4669a4ec92c9ac9f8b3b (table test-table [id=1ccd433cff864676bd7883729872e450]) successfully deleted
I20260929 16:09:35.103082 8100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:35.196026 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 7977
I20260929 16:09:35.214530 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 8111
Sep 29 16:09:35 dist-test-slave-x07d krb5kdc[7871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.130@KRBTEST.COM
I20260929 16:09:35.231834 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:35.215440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.20.121.130:37977)
0929 16:09:35.215635 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:35.215640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:35.215663 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:09:35.215870 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:35.215876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:35.215967 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:35.216088 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:35.217496 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:35.218366 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:35.219760 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:35.220174 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:35.220321 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:35.224867 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:35.224885 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:35.224902 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:35.224956 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:35.228449 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:35.229331 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:35.229340 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:35.229342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:35.229484 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:35.229899 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:35.229905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:35.229910 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:35.230525 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:35.230855 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:35.231198 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 16:09:35.231459 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 16:09:35.232507 8234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:35.331859 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 8111
I20260929 16:09:35.349654 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 8245
Sep 29 16:09:35 dist-test-slave-x07d krb5kdc[7871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.131@KRBTEST.COM
I20260929 16:09:35.369050 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:35.350546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45826 (local address 127.20.121.131:38539)
0929 16:09:35.350787 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:35.350794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:35.350824 (+ 30us) server_negotiation.cc:407] Connection header received
0929 16:09:35.350904 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:35.350911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:35.350996 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:35.351134 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:35.352426 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:35.353661 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:35.355152 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:35.355878 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:35.356047 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:35.360889 (+ 4842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:35.360914 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:35.360937 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:35.361002 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:35.365620 (+ 4618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:35.366565 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:35.366578 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:35.366583 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:35.366774 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:35.367189 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:35.367195 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:35.367198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:35.367705 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:35.368061 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:35.368358 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 16:09:35.368637 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260929 16:09:35.369961 8368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:35.498780 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 8245
I20260929 16:09:35.521190 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 7884
I20260929 16:09:35.522496 7946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:35.637346 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 7884
2026-09-29T16:09:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4748 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T16:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:35Z Disabled control of system clock
I20260929 16:09:35.721347 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37109
--webserver_interface=127.20.121.190
--webserver_port=0
--builtin_ntp_servers=127.20.121.148:41613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:37109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 16:09:36.071029 8426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:36.071337 8426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:36.071419 8426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:36.078989 8426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:36.079150 8426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:36.079216 8426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:36.079308 8426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:36.079371 8426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:36.090569 8426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:41613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:37109
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:36.091840 8426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:36.094079 8426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:36.107326 8431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:36.107259 8434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:36.107290 8432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:36.108712 8426 server_base.cc:1034] running on GCE node
I20260929 16:09:36.109642 8426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:36.111292 8426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:36.112524 8426 hybrid_clock.cc:648] HybridClock initialized: now 1790698176112476 us; error 39 us; skew 500 ppm
I20260929 16:09:36.112944 8426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:36.115803 8426 webserver.cc:460] Webserver started at http://127.20.121.190:38269/ using document root <none> and password file <none>
I20260929 16:09:36.116611 8426 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:36.116712 8426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:36.117070 8426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:36.120509 8426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "ec736d4d0e4f41a1bc586225fea74d4d"
format_stamp: "Formatted at 2026-09-29 16:09:36 on dist-test-slave-x07d"
server_key: "215b06b06ce39e0151b9113abcfa6877"
server_key_iv: "3c5dbd30aec7e5827e05243240a6b9d7"
server_key_version: "encryptionkey@0"
I20260929 16:09:36.121460 8426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "ec736d4d0e4f41a1bc586225fea74d4d"
format_stamp: "Formatted at 2026-09-29 16:09:36 on dist-test-slave-x07d"
server_key: "215b06b06ce39e0151b9113abcfa6877"
server_key_iv: "3c5dbd30aec7e5827e05243240a6b9d7"
server_key_version: "encryptionkey@0"
I20260929 16:09:36.127775 8426 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260929 16:09:36.132138 8440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:36.134164 8426 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:09:36.134402 8426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "ec736d4d0e4f41a1bc586225fea74d4d"
format_stamp: "Formatted at 2026-09-29 16:09:36 on dist-test-slave-x07d"
server_key: "215b06b06ce39e0151b9113abcfa6877"
server_key_iv: "3c5dbd30aec7e5827e05243240a6b9d7"
server_key_version: "encryptionkey@0"
I20260929 16:09:36.134647 8426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:36.161397 8426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:36.162513 8426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:36.162864 8426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:36.184867 8426 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:37109
I20260929 16:09:36.184893 8491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:37109 every 8 connection(s)
I20260929 16:09:36.186512 8426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:36.189481 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 8426
I20260929 16:09:36.189680 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:36.190302 20966 external_mini_cluster.cc:1428] Setting key 0b712c9a46c9b42b7b933b1096d0425d
I20260929 16:09:36.191330 8492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:36.204923 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:36.192397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34612 (local address 127.20.121.190:37109)
0929 16:09:36.193528 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:36.193540 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:36.193584 (+ 44us) server_negotiation.cc:407] Connection header received
0929 16:09:36.195256 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:36.195283 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:36.195825 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:36.196295 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:36.196907 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:36.198370 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:36.199748 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:36.200445 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:36.200640 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:36.202059 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:36.202077 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:36.202095 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:36.202754 (+ 659us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:36.202841 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:36.202867 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:36.202961 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:36.203377 (+ 416us) server_negotiation.cc:299] Negotiation successful
0929 16:09:36.203815 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":445,"threads_started":1}
I20260929 16:09:36.205435 8492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d: Bootstrap starting.
I20260929 16:09:36.208734 8492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:36.209785 8492 log.cc:824] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:36.213119 8492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d: No bootstrap required, opened a new log
I20260929 16:09:36.223220 8492 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec736d4d0e4f41a1bc586225fea74d4d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37109 } }
I20260929 16:09:36.223706 8492 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:36.223814 8492 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec736d4d0e4f41a1bc586225fea74d4d, State: Initialized, Role: FOLLOWER
I20260929 16:09:36.224443 8492 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec736d4d0e4f41a1bc586225fea74d4d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37109 } }
I20260929 16:09:36.224690 8492 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:36.224838 8492 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:36.225011 8492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:36.227715 8492 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec736d4d0e4f41a1bc586225fea74d4d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37109 } }
I20260929 16:09:36.228376 8492 leader_election.cc:302] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec736d4d0e4f41a1bc586225fea74d4d; no voters:
I20260929 16:09:36.229095 8492 leader_election.cc:288] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:36.229323 8497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:36.230257 8497 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [term 1 LEADER]: Becoming Leader. State: Replica: ec736d4d0e4f41a1bc586225fea74d4d, State: Running, Role: LEADER
I20260929 16:09:36.230835 8497 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec736d4d0e4f41a1bc586225fea74d4d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37109 } }
I20260929 16:09:36.231405 8492 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:36.233763 8499 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec736d4d0e4f41a1bc586225fea74d4d. Latest consensus state: current_term: 1 leader_uuid: "ec736d4d0e4f41a1bc586225fea74d4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec736d4d0e4f41a1bc586225fea74d4d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37109 } } }
I20260929 16:09:36.234311 8499 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:36.234207 8498 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec736d4d0e4f41a1bc586225fea74d4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec736d4d0e4f41a1bc586225fea74d4d" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37109 } } }
I20260929 16:09:36.234541 8498 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:36.235387 8504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:36.245609 8504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:36.257913 8504 catalog_manager.cc:1347] Generated new cluster ID: 8dad0a0c45974805b38b8854ab79daa3
I20260929 16:09:36.258090 8504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:36.271450 8504 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:36.272910 8504 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:36.281050 8504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d: Generated new TSK 0
I20260929 16:09:36.282248 8504 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:36.295361 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.129:0
--local_ip_for_outbound_sockets=127.20.121.129
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37109
--builtin_ntp_servers=127.20.121.148:41613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 16:09:36.647465 8516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:36.647778 8516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:36.647864 8516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:36.655336 8516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:36.655467 8516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:36.655578 8516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.129
I20260929 16:09:36.665021 8516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:41613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.121.129
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.129
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:36.666352 8516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:36.668339 8516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:36.682513 8522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:36.682554 8521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:36.682482 8524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:36.683054 8516 server_base.cc:1034] running on GCE node
I20260929 16:09:36.683972 8516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:36.684970 8516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:36.686257 8516 hybrid_clock.cc:648] HybridClock initialized: now 1790698176686210 us; error 31 us; skew 500 ppm
I20260929 16:09:36.686627 8516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:36.688934 8516 webserver.cc:460] Webserver started at http://127.20.121.129:35421/ using document root <none> and password file <none>
I20260929 16:09:36.689663 8516 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:36.689798 8516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:36.690196 8516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:36.693203 8516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/data/instance:
uuid: "2eff11aae3b941c390799581030ae587"
format_stamp: "Formatted at 2026-09-29 16:09:36 on dist-test-slave-x07d"
server_key: "8111ea6b911bd28ff0ab25bf3d9859e0"
server_key_iv: "a1103be48c913eb5060872672025f0ee"
server_key_version: "encryptionkey@0"
I20260929 16:09:36.694099 8516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance:
uuid: "2eff11aae3b941c390799581030ae587"
format_stamp: "Formatted at 2026-09-29 16:09:36 on dist-test-slave-x07d"
server_key: "8111ea6b911bd28ff0ab25bf3d9859e0"
server_key_iv: "a1103be48c913eb5060872672025f0ee"
server_key_version: "encryptionkey@0"
I20260929 16:09:36.699555 8516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 16:09:36.703447 8530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:36.705320 8516 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 16:09:36.705576 8516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/wal
uuid: "2eff11aae3b941c390799581030ae587"
format_stamp: "Formatted at 2026-09-29 16:09:36 on dist-test-slave-x07d"
server_key: "8111ea6b911bd28ff0ab25bf3d9859e0"
server_key_iv: "a1103be48c913eb5060872672025f0ee"
server_key_version: "encryptionkey@0"
I20260929 16:09:36.705827 8516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:36.721516 8516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:36.722518 8516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:36.722838 8516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:36.723956 8516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:36.725493 8516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:36.725586 8516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:36.725742 8516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:36.725819 8516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:36.761174 8516 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.129:38631
I20260929 16:09:36.761228 8642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.129:38631 every 8 connection(s)
I20260929 16:09:36.762667 8516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/data/info.pb
I20260929 16:09:36.771939 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 8516
I20260929 16:09:36.772130 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/wal/instance
I20260929 16:09:36.772593 20966 external_mini_cluster.cc:1428] Setting key ab3bc041bb31f8a5da810f9517b273ca
I20260929 16:09:36.775810 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.130:0
--local_ip_for_outbound_sockets=127.20.121.130
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37109
--builtin_ntp_servers=127.20.121.148:41613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 16:09:36.776728 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:36.765679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:51029 (local address 127.20.121.190:37109)
0929 16:09:36.766195 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:36.766199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:36.767451 (+ 1252us) server_negotiation.cc:407] Connection header received
0929 16:09:36.769363 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:36.769370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:36.769469 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:36.769597 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:36.770832 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:36.771603 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:36.773394 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:36.774002 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:36.774154 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:36.775816 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:36.775821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:36.775825 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:36.775845 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:36.775886 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:36.775898 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:36.776005 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:36.776257 (+ 252us) server_negotiation.cc:299] Negotiation successful
0929 16:09:36.776374 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":410,"thread_start_us":252,"threads_started":1}
I20260929 16:09:36.777910 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:36.766356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37109 (local address 127.20.121.129:51029)
0929 16:09:36.767269 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:36.767315 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:36.769019 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:36.769869 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:36.769878 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:36.769977 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:36.770649 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:36.770666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:36.771768 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:36.771773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:36.773234 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:36.773248 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:36.774246 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:36.774252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:36.774588 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:36.775412 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:09:36.775418 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:36.775479 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:09:36.775596 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:36.776008 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:36.776022 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:36.776032 (+ 10us) client_negotiation.cc:769] Sending connection context
0929 16:09:36.776249 (+ 217us) client_negotiation.cc:240] Negotiation successful
0929 16:09:36.776699 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":263,"threads_started":1}
I20260929 16:09:36.779599 8643 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37109
I20260929 16:09:36.779932 8643 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:36.780781 8643 heartbeater.cc:511] Master 127.20.121.190:37109 requested a full tablet report, sending...
I20260929 16:09:36.783309 8457 ts_manager.cc:194] Registered new tserver with Master: 2eff11aae3b941c390799581030ae587 (127.20.121.129:38631)
I20260929 16:09:36.785264 8457 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.129:51029
W20260929 16:09:37.125838 8647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:37.126201 8647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:37.126299 8647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:37.134982 8647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:37.135118 8647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:37.135241 8647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.130
I20260929 16:09:37.146832 8647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:41613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.121.130
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.130
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:37.148259 8647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:37.150192 8647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:37.165788 8652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:37.165757 8653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:37.165917 8655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:37.166898 8647 server_base.cc:1034] running on GCE node
I20260929 16:09:37.167640 8647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:37.168686 8647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:37.169935 8647 hybrid_clock.cc:648] HybridClock initialized: now 1790698177169899 us; error 31 us; skew 500 ppm
I20260929 16:09:37.170318 8647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:37.172920 8647 webserver.cc:460] Webserver started at http://127.20.121.130:39259/ using document root <none> and password file <none>
I20260929 16:09:37.173595 8647 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:37.173695 8647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:37.173967 8647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:37.176990 8647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/data/instance:
uuid: "52ced56cac074831964d4a9f07576ef0"
format_stamp: "Formatted at 2026-09-29 16:09:37 on dist-test-slave-x07d"
server_key: "6f5d0084f82bb04646908a1dca578e46"
server_key_iv: "885c19516191980c4819b3e0c14581f7"
server_key_version: "encryptionkey@0"
I20260929 16:09:37.177718 8647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance:
uuid: "52ced56cac074831964d4a9f07576ef0"
format_stamp: "Formatted at 2026-09-29 16:09:37 on dist-test-slave-x07d"
server_key: "6f5d0084f82bb04646908a1dca578e46"
server_key_iv: "885c19516191980c4819b3e0c14581f7"
server_key_version: "encryptionkey@0"
I20260929 16:09:37.183151 8647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260929 16:09:37.187539 8661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:37.189296 8647 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 16:09:37.189489 8647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/wal
uuid: "52ced56cac074831964d4a9f07576ef0"
format_stamp: "Formatted at 2026-09-29 16:09:37 on dist-test-slave-x07d"
server_key: "6f5d0084f82bb04646908a1dca578e46"
server_key_iv: "885c19516191980c4819b3e0c14581f7"
server_key_version: "encryptionkey@0"
I20260929 16:09:37.189745 8647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:37.208808 8647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:37.209654 8647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:37.209987 8647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:37.211126 8647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:37.212710 8647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:37.212800 8647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:37.212984 8647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:37.213053 8647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:37.248117 8647 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.130:44911
I20260929 16:09:37.248159 8773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.130:44911 every 8 connection(s)
I20260929 16:09:37.249511 8647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/data/info.pb
I20260929 16:09:37.254468 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 8647
I20260929 16:09:37.254652 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/wal/instance
I20260929 16:09:37.255105 20966 external_mini_cluster.cc:1428] Setting key 45772aaed2019a6c6cbaa037e07da46c
I20260929 16:09:37.258317 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.121.131:0
--local_ip_for_outbound_sockets=127.20.121.131
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37109
--builtin_ntp_servers=127.20.121.148:41613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 16:09:37.264590 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.252312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.130:48635 (local address 127.20.121.190:37109)
0929 16:09:37.252537 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:37.252543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:37.254258 (+ 1715us) server_negotiation.cc:407] Connection header received
0929 16:09:37.256322 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:37.256327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:37.256402 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:37.256519 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:37.257902 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.259079 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.260911 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.261609 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.261752 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.263505 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:37.263514 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:37.263523 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:37.263552 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:37.263597 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:37.263617 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:37.263715 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:37.264051 (+ 336us) server_negotiation.cc:299] Negotiation successful
0929 16:09:37.264170 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 16:09:37.265667 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.252925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37109 (local address 127.20.121.130:48635)
0929 16:09:37.254001 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:37.254106 (+ 105us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:37.256050 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:37.256790 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:37.256799 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:37.256896 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:37.257617 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.257635 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.259283 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.259288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.260740 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.260755 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.261907 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.261912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.262239 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.263117 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:09:37.263120 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:37.263166 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:09:37.263265 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:37.263791 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:37.263802 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:37.263809 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 16:09:37.264002 (+ 193us) client_negotiation.cc:240] Negotiation successful
0929 16:09:37.264374 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":346,"threads_started":1}
I20260929 16:09:37.267465 8774 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37109
I20260929 16:09:37.267856 8774 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:37.268783 8774 heartbeater.cc:511] Master 127.20.121.190:37109 requested a full tablet report, sending...
I20260929 16:09:37.270562 8457 ts_manager.cc:194] Registered new tserver with Master: 52ced56cac074831964d4a9f07576ef0 (127.20.121.130:44911)
I20260929 16:09:37.271530 8457 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.130:48635
W20260929 16:09:37.601593 8777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:37.601899 8777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:37.602074 8777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:37.609795 8777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:37.609925 8777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:37.610131 8777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.121.131
I20260929 16:09:37.619736 8777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:41613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.121.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.121.131
--webserver_port=0
--tserver_master_addrs=127.20.121.190:37109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.121.131
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:37.621122 8777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:37.623195 8777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:37.637152 8783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:37.637234 8782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:37.637207 8785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:37.638276 8777 server_base.cc:1034] running on GCE node
I20260929 16:09:37.639019 8777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:37.640151 8777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:37.641397 8777 hybrid_clock.cc:648] HybridClock initialized: now 1790698177641320 us; error 64 us; skew 500 ppm
I20260929 16:09:37.641762 8777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 16:09:37.644518 8777 webserver.cc:460] Webserver started at http://127.20.121.131:46015/ using document root <none> and password file <none>
I20260929 16:09:37.645390 8777 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:37.645527 8777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:37.645882 8777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:37.649036 8777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/data/instance:
uuid: "90892553dc304caebcd8506b9724f92d"
format_stamp: "Formatted at 2026-09-29 16:09:37 on dist-test-slave-x07d"
server_key: "518ae7e4415c00b94418fe346fb52635"
server_key_iv: "ca85e9f6484685d1b9fb1c0c19373e2b"
server_key_version: "encryptionkey@0"
I20260929 16:09:37.649931 8777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance:
uuid: "90892553dc304caebcd8506b9724f92d"
format_stamp: "Formatted at 2026-09-29 16:09:37 on dist-test-slave-x07d"
server_key: "518ae7e4415c00b94418fe346fb52635"
server_key_iv: "ca85e9f6484685d1b9fb1c0c19373e2b"
server_key_version: "encryptionkey@0"
I20260929 16:09:37.655776 8777 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 16:09:37.659580 8791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:37.661038 8777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:09:37.661237 8777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/wal
uuid: "90892553dc304caebcd8506b9724f92d"
format_stamp: "Formatted at 2026-09-29 16:09:37 on dist-test-slave-x07d"
server_key: "518ae7e4415c00b94418fe346fb52635"
server_key_iv: "ca85e9f6484685d1b9fb1c0c19373e2b"
server_key_version: "encryptionkey@0"
I20260929 16:09:37.661476 8777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:37.678926 8777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:37.679813 8777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:37.680140 8777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:37.681190 8777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 16:09:37.682883 8777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 16:09:37.683001 8777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:37.683159 8777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 16:09:37.683233 8777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:37.718408 8777 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.131:36867
I20260929 16:09:37.718488 8903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.131:36867 every 8 connection(s)
I20260929 16:09:37.719815 8777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/data/info.pb
I20260929 16:09:37.728506 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 8777
I20260929 16:09:37.728727 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/wal/instance
I20260929 16:09:37.729179 20966 external_mini_cluster.cc:1428] Setting key 7ba0cdce6b762a936e32d41e459f0c1f
I20260929 16:09:37.733264 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.722381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.131:48175 (local address 127.20.121.190:37109)
0929 16:09:37.722589 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:37.722594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:37.723988 (+ 1394us) server_negotiation.cc:407] Connection header received
0929 16:09:37.725832 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:37.725838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:37.725950 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:37.726096 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:37.727217 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.727973 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.729730 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.730371 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.730495 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.732213 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:37.732219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:37.732223 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:37.732266 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:37.732314 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:37.732332 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:37.732459 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:37.732789 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 16:09:37.732885 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 16:09:37.734278 8906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.722861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:37109 (local address 127.20.121.131:48175)
0929 16:09:37.723799 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:37.723841 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:37.725522 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:37.726337 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:37.726346 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:37.726445 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:09:37.727032 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.727048 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.728147 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.728151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.729580 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.729593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.730607 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.730613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.730906 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.731748 (+ 842us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 16:09:37.731752 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:09:37.731810 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 16:09:37.732003 (+ 193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:09:37.732500 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:09:37.732508 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:09:37.732515 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 16:09:37.732689 (+ 174us) client_negotiation.cc:240] Negotiation successful
0929 16:09:37.733059 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":261,"threads_started":1}
I20260929 16:09:37.735780 8904 heartbeater.cc:347] Connected to a master server at 127.20.121.190:37109
I20260929 16:09:37.736308 8904 heartbeater.cc:464] Registering TS with master...
I20260929 16:09:37.737128 8904 heartbeater.cc:511] Master 127.20.121.190:37109 requested a full tablet report, sending...
I20260929 16:09:37.738641 8457 ts_manager.cc:194] Registered new tserver with Master: 90892553dc304caebcd8506b9724f92d (127.20.121.131:36867)
I20260929 16:09:37.739549 8457 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.121.131:48175
I20260929 16:09:37.744894 20966 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 16:09:37.755367 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.749215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34624 (local address 127.20.121.190:37109)
0929 16:09:37.749446 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:37.749451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:37.749831 (+ 380us) server_negotiation.cc:407] Connection header received
0929 16:09:37.749978 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:37.749982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:37.750116 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:37.750207 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:37.750666 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.751387 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.752765 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.753213 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.753319 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.754706 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:37.754711 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:37.754714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:37.754734 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:37.754768 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:37.754780 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:37.754853 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:37.755054 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 16:09:37.755139 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 16:09:37.759523 8457 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 16:09:37.766780 8457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 16:09:37.788728 8643 heartbeater.cc:503] Master 127.20.121.190:37109 was elected leader, sending a full tablet report...
I20260929 16:09:37.813764 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.779251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.129:38631 (local address 127.0.0.1:54186)
0929 16:09:37.798711 (+ 19460us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:37.798787 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:37.799035 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:37.800576 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:37.800589 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:37.800637 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:37.801186 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.801205 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.805981 (+ 4776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.805991 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.810187 (+ 4196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.810205 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.812509 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.812516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.812901 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.812977 (+ 76us) client_negotiation.cc:769] Sending connection context
0929 16:09:37.813118 (+ 141us) client_negotiation.cc:240] Negotiation successful
0929 16:09:37.813334 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19054,"mutex_wait_us":99,"thread_start_us":28490,"threads_started":1}
I20260929 16:09:37.813814 8918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.780238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54186 (local address 127.20.121.129:38631)
0929 16:09:37.794873 (+ 14635us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:37.794890 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:37.799322 (+ 4432us) server_negotiation.cc:407] Connection header received
0929 16:09:37.799502 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:37.799516 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:37.799885 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:37.800130 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:37.801506 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.805714 (+ 4208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.810478 (+ 4764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.812287 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.812515 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.812775 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:37.813197 (+ 422us) server_negotiation.cc:299] Negotiation successful
0929 16:09:37.813402 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14359,"thread_start_us":13825,"threads_started":1}
I20260929 16:09:37.816974 8919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.779629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:36867 (local address 127.0.0.1:50406)
0929 16:09:37.807392 (+ 27763us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:37.807415 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:37.807643 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:37.808885 (+ 1242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:37.808890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:37.808919 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:37.809465 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.809481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.811636 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.811643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.814184 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.814198 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.816010 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.816018 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.816212 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.816248 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 16:09:37.816364 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 16:09:37.816532 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27582,"mutex_wait_us":72,"thread_start_us":15949,"threads_started":1}
I20260929 16:09:37.817000 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.780971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50406 (local address 127.20.121.131:36867)
0929 16:09:37.806791 (+ 25820us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:37.806803 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:37.807545 (+ 742us) server_negotiation.cc:407] Connection header received
0929 16:09:37.808025 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:37.808035 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:37.808343 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:37.808577 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:37.809688 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.811489 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.814508 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.815787 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.816034 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.816255 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:37.816429 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 16:09:37.816608 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25572,"thread_start_us":25358,"threads_started":1}
I20260929 16:09:37.817116 8916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.780208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:44911 (local address 127.0.0.1:55850)
0929 16:09:37.808064 (+ 27856us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:37.808082 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:37.808296 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:37.809199 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:37.809203 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:37.809232 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:37.809877 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.809891 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.811744 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.811753 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.814400 (+ 2647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.814420 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.816293 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.816300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.816511 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.816543 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 16:09:37.816635 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 16:09:37.816761 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27729,"mutex_wait_us":118,"thread_start_us":26204,"threads_started":1}
I20260929 16:09:37.817786 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.780870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55850 (local address 127.20.121.130:44911)
0929 16:09:37.805554 (+ 24684us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:37.805567 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:37.808189 (+ 2622us) server_negotiation.cc:407] Connection header received
0929 16:09:37.808511 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:37.808519 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:37.808727 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:37.808899 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:37.810131 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.811624 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.814580 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.815757 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.815916 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.816146 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:37.817267 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0929 16:09:37.817460 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24421,"thread_start_us":24707,"threads_started":1}
I20260929 16:09:37.818553 8578 tablet_service.cc:1467] Processing CreateTablet for tablet 4075d99f9b6d41ad9e690d4e6f187ef2 (DEFAULT_TABLE table=test-table [id=4087d2e864f34b95a8dc0758de0e747a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:37.820326 8578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4075d99f9b6d41ad9e690d4e6f187ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:37.821383 8709 tablet_service.cc:1467] Processing CreateTablet for tablet 4075d99f9b6d41ad9e690d4e6f187ef2 (DEFAULT_TABLE table=test-table [id=4087d2e864f34b95a8dc0758de0e747a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:37.821370 8839 tablet_service.cc:1467] Processing CreateTablet for tablet 4075d99f9b6d41ad9e690d4e6f187ef2 (DEFAULT_TABLE table=test-table [id=4087d2e864f34b95a8dc0758de0e747a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 16:09:37.822921 8839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4075d99f9b6d41ad9e690d4e6f187ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:37.822921 8709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4075d99f9b6d41ad9e690d4e6f187ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:37.832468 8922 tablet_bootstrap.cc:492] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: Bootstrap starting.
I20260929 16:09:37.836578 8922 tablet_bootstrap.cc:654] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:37.837776 8922 log.cc:824] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:37.840082 8925 tablet_bootstrap.cc:492] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: Bootstrap starting.
I20260929 16:09:37.841414 8922 tablet_bootstrap.cc:492] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: No bootstrap required, opened a new log
I20260929 16:09:37.841735 8922 ts_tablet_manager.cc:1397] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260929 16:09:37.842798 8924 tablet_bootstrap.cc:492] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: Bootstrap starting.
I20260929 16:09:37.844146 8925 tablet_bootstrap.cc:654] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:37.845482 8925 log.cc:824] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:37.846675 8924 tablet_bootstrap.cc:654] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:37.847994 8924 log.cc:824] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:37.850847 8925 tablet_bootstrap.cc:492] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: No bootstrap required, opened a new log
I20260929 16:09:37.850883 8924 tablet_bootstrap.cc:492] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: No bootstrap required, opened a new log
I20260929 16:09:37.851171 8924 ts_tablet_manager.cc:1397] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 16:09:37.851173 8925 ts_tablet_manager.cc:1397] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260929 16:09:37.852742 8922 raft_consensus.cc:348] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eff11aae3b941c390799581030ae587" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38631 } } peers { permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 } } peers { permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 } }
I20260929 16:09:37.853230 8922 raft_consensus.cc:374] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:37.853386 8922 raft_consensus.cc:729] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eff11aae3b941c390799581030ae587, State: Initialized, Role: FOLLOWER
I20260929 16:09:37.854113 8922 consensus_queue.cc:260] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eff11aae3b941c390799581030ae587" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38631 } } peers { permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 } } peers { permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 } }
I20260929 16:09:37.856109 8922 ts_tablet_manager.cc:1428] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260929 16:09:37.860805 8924 raft_consensus.cc:348] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eff11aae3b941c390799581030ae587" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38631 } } peers { permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 } } peers { permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 } }
I20260929 16:09:37.861392 8924 raft_consensus.cc:374] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:37.861559 8924 raft_consensus.cc:729] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ced56cac074831964d4a9f07576ef0, State: Initialized, Role: FOLLOWER
I20260929 16:09:37.861433 8925 raft_consensus.cc:348] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eff11aae3b941c390799581030ae587" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38631 } } peers { permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 } } peers { permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 } }
I20260929 16:09:37.861937 8925 raft_consensus.cc:374] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:37.862077 8925 raft_consensus.cc:729] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90892553dc304caebcd8506b9724f92d, State: Initialized, Role: FOLLOWER
I20260929 16:09:37.862455 8924 consensus_queue.cc:260] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eff11aae3b941c390799581030ae587" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38631 } } peers { permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 } } peers { permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 } }
I20260929 16:09:37.862649 8925 consensus_queue.cc:260] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eff11aae3b941c390799581030ae587" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38631 } } peers { permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 } } peers { permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 } }
I20260929 16:09:37.863982 8774 heartbeater.cc:503] Master 127.20.121.190:37109 was elected leader, sending a full tablet report...
I20260929 16:09:37.864274 8904 heartbeater.cc:503] Master 127.20.121.190:37109 was elected leader, sending a full tablet report...
I20260929 16:09:37.864503 8924 ts_tablet_manager.cc:1428] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260929 16:09:37.867170 8925 ts_tablet_manager.cc:1428] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: Time spent starting tablet: real 0.016s user 0.006s sys 0.007s
I20260929 16:09:37.923237 8928 raft_consensus.cc:482] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 16:09:37.923705 8928 raft_consensus.cc:504] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eff11aae3b941c390799581030ae587" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38631 } } peers { permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 } } peers { permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 } }
I20260929 16:09:37.925652 8928 leader_election.cc:288] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90892553dc304caebcd8506b9724f92d (127.20.121.131:36867), 52ced56cac074831964d4a9f07576ef0 (127.20.121.130:44911)
I20260929 16:09:37.935578 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.926334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:42615 (local address 127.20.121.130:44911)
0929 16:09:37.926538 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:37.926546 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:37.927177 (+ 631us) server_negotiation.cc:407] Connection header received
0929 16:09:37.927382 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:37.927389 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:37.927478 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:37.927604 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:37.928278 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.929057 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.932757 (+ 3700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.934449 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.934677 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.934760 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:37.935107 (+ 347us) server_negotiation.cc:299] Negotiation successful
0929 16:09:37.935263 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 16:09:37.935681 8931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.926100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.130:44911 (local address 127.20.121.129:42615)
0929 16:09:37.927023 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:37.927042 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:37.927199 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:37.927637 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:37.927642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:37.927672 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:37.928111 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.928122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.929216 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.929225 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.932045 (+ 2820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.932067 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.934695 (+ 2628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.934702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.934910 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.934938 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 16:09:37.935071 (+ 133us) client_negotiation.cc:240] Negotiation successful
0929 16:09:37.935241 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":7,"thread_start_us":479,"threads_started":1}
I20260929 16:09:37.936896 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.926189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.121.129:53665 (local address 127.20.121.131:36867)
0929 16:09:37.926455 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:37.926462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:37.927914 (+ 1452us) server_negotiation.cc:407] Connection header received
0929 16:09:37.928153 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:37.928161 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:37.928234 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:37.928357 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 16:09:37.928938 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.929702 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.933005 (+ 3303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.935356 (+ 2351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.935577 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.935657 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:37.936433 (+ 776us) server_negotiation.cc:299] Negotiation successful
0929 16:09:37.936615 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 16:09:37.937315 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.926915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.131:36867 (local address 127.20.121.129:53665)
0929 16:09:37.927783 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0929 16:09:37.927801 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:09:37.927991 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:09:37.928372 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:09:37.928377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:09:37.928402 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 16:09:37.928751 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.928761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.929878 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.929886 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.932726 (+ 2840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:09:37.932753 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.935833 (+ 3080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.935844 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:09:37.936106 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.936158 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 16:09:37.936289 (+ 131us) client_negotiation.cc:240] Negotiation successful
0929 16:09:37.936491 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":10,"thread_start_us":497,"threads_started":1}
I20260929 16:09:37.938468 8729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4075d99f9b6d41ad9e690d4e6f187ef2" candidate_uuid: "2eff11aae3b941c390799581030ae587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ced56cac074831964d4a9f07576ef0" is_pre_election: true
I20260929 16:09:37.939159 8729 raft_consensus.cc:2454] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2eff11aae3b941c390799581030ae587 in term 0.
I20260929 16:09:37.940426 8531 leader_election.cc:302] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2eff11aae3b941c390799581030ae587, 52ced56cac074831964d4a9f07576ef0; no voters:
I20260929 16:09:37.941103 8928 raft_consensus.cc:2781] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 16:09:37.941283 8928 raft_consensus.cc:482] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 16:09:37.941407 8928 raft_consensus.cc:3037] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:37.941354 8859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4075d99f9b6d41ad9e690d4e6f187ef2" candidate_uuid: "2eff11aae3b941c390799581030ae587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90892553dc304caebcd8506b9724f92d" is_pre_election: true
I20260929 16:09:37.942067 8859 raft_consensus.cc:2454] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2eff11aae3b941c390799581030ae587 in term 0.
I20260929 16:09:37.944366 8928 raft_consensus.cc:504] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eff11aae3b941c390799581030ae587" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38631 } } peers { permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 } } peers { permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 } }
I20260929 16:09:37.945212 8928 leader_election.cc:288] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [CANDIDATE]: Term 1 election: Requested vote from peers 90892553dc304caebcd8506b9724f92d (127.20.121.131:36867), 52ced56cac074831964d4a9f07576ef0 (127.20.121.130:44911)
I20260929 16:09:37.945827 8859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4075d99f9b6d41ad9e690d4e6f187ef2" candidate_uuid: "2eff11aae3b941c390799581030ae587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90892553dc304caebcd8506b9724f92d"
I20260929 16:09:37.946036 8729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4075d99f9b6d41ad9e690d4e6f187ef2" candidate_uuid: "2eff11aae3b941c390799581030ae587" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ced56cac074831964d4a9f07576ef0"
I20260929 16:09:37.946215 8859 raft_consensus.cc:3037] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:37.946282 8729 raft_consensus.cc:3037] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:37.948860 8729 raft_consensus.cc:2454] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2eff11aae3b941c390799581030ae587 in term 1.
I20260929 16:09:37.949159 8859 raft_consensus.cc:2454] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2eff11aae3b941c390799581030ae587 in term 1.
I20260929 16:09:37.949343 8531 leader_election.cc:302] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2eff11aae3b941c390799581030ae587, 52ced56cac074831964d4a9f07576ef0; no voters:
I20260929 16:09:37.949690 8928 raft_consensus.cc:2781] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:37.950389 8928 raft_consensus.cc:686] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 1 LEADER]: Becoming Leader. State: Replica: 2eff11aae3b941c390799581030ae587, State: Running, Role: LEADER
I20260929 16:09:37.951511 8928 consensus_queue.cc:237] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eff11aae3b941c390799581030ae587" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38631 } } peers { permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 } } peers { permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 } }
I20260929 16:09:37.957190 8456 catalog_manager.cc:5495] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2eff11aae3b941c390799581030ae587 (127.20.121.129). New cstate: current_term: 1 leader_uuid: "2eff11aae3b941c390799581030ae587" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eff11aae3b941c390799581030ae587" member_type: VOTER last_known_addr { host: "127.20.121.129" port: 38631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 } health_report { overall_health: UNKNOWN } } }
W20260929 16:09:37.971581 8905 tablet.cc:2369] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:09:37.993130 8918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:37.987395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.20.121.129:38631)
0929 16:09:37.987597 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:37.987603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:37.987624 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:09:37.987946 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:37.987950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:37.988057 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:37.988229 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:09:37.988633 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.989375 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.990861 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:37.991305 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:37.991447 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:37.991986 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:09:37.992533 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:09:37.992697 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:37.992764 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 16:09:37.992859 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260929 16:09:38.001901 8775 tablet.cc:2369] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:09:38.003767 8729 raft_consensus.cc:1261] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0 [term 1 FOLLOWER]: Refusing update from remote peer 2eff11aae3b941c390799581030ae587: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:38.003768 8859 raft_consensus.cc:1261] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d [term 1 FOLLOWER]: Refusing update from remote peer 2eff11aae3b941c390799581030ae587: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:09:38.005131 8933 consensus_queue.cc:1035] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ced56cac074831964d4a9f07576ef0" member_type: VOTER last_known_addr { host: "127.20.121.130" port: 44911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:09:38.005628 8928 consensus_queue.cc:1035] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90892553dc304caebcd8506b9724f92d" member_type: VOTER last_known_addr { host: "127.20.121.131" port: 36867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 16:09:38.015619 8644 tablet.cc:2369] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:09:38.021950 8936 mvcc.cc:205] Tried to move back new op lower bound from 7334699737089998848 to 7334699736893927424. Current Snapshot: MvccSnapshot[applied={T|T < 7334699737089998848}]
I20260929 16:09:38.025069 8937 mvcc.cc:205] Tried to move back new op lower bound from 7334699737089998848 to 7334699736893927424. Current Snapshot: MvccSnapshot[applied={T|T < 7334699737089998848}]
I20260929 16:09:38.034858 8456 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:38.035226 8456 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 16:09:38.039813 8456 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec736d4d0e4f41a1bc586225fea74d4d: Sending DeleteTablet for 3 replicas of tablet 4075d99f9b6d41ad9e690d4e6f187ef2
I20260929 16:09:38.041304 8578 tablet_service.cc:1514] Processing DeleteTablet for tablet 4075d99f9b6d41ad9e690d4e6f187ef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:38 UTC) from {username='slave'} at 127.0.0.1:54186
I20260929 16:09:38.041260 8839 tablet_service.cc:1514] Processing DeleteTablet for tablet 4075d99f9b6d41ad9e690d4e6f187ef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:38 UTC) from {username='slave'} at 127.0.0.1:50406
I20260929 16:09:38.041425 8709 tablet_service.cc:1514] Processing DeleteTablet for tablet 4075d99f9b6d41ad9e690d4e6f187ef2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 16:09:38 UTC) from {username='slave'} at 127.0.0.1:55850
I20260929 16:09:38.051574 8947 tablet_replica.cc:330] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: stopping tablet replica
I20260929 16:09:38.051689 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 8516
I20260929 16:09:38.052614 8946 tablet_replica.cc:330] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: stopping tablet replica
I20260929 16:09:38.053611 8945 tablet_replica.cc:330] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: stopping tablet replica
I20260929 16:09:38.054124 8945 raft_consensus.cc:2229] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 1 LEADER]: Raft consensus shutting down.
I20260929 16:09:38.054937 8945 raft_consensus.cc:2258] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:38.055644 8946 raft_consensus.cc:2229] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:38.056200 8946 raft_consensus.cc:2258] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:38.058727 8945 ts_tablet_manager.cc:1905] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:38.058761 8947 raft_consensus.cc:2229] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 16:09:38.059312 8947 raft_consensus.cc:2258] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 16:09:38.060628 8946 ts_tablet_manager.cc:1905] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:38.063885 8947 ts_tablet_manager.cc:1905] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 16:09:38.066946 8945 ts_tablet_manager.cc:1918] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:38.067144 8945 log.cc:1196] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-0/wal/wals/4075d99f9b6d41ad9e690d4e6f187ef2
I20260929 16:09:38.067745 8945 ts_tablet_manager.cc:1939] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 2eff11aae3b941c390799581030ae587: Deleting consensus metadata
I20260929 16:09:38.068907 8918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:38.052938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.20.121.129:38631)
0929 16:09:38.053132 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:38.053139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:38.060230 (+ 7091us) server_negotiation.cc:407] Connection header received
0929 16:09:38.060553 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:38.060560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:38.060682 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:38.060781 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:38.061482 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:38.062792 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:38.064598 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:38.065326 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:38.065422 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:38.067057 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:38.067069 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:38.067089 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:38.067959 (+ 870us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:38.068027 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:38.068051 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:38.068126 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:38.068354 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 16:09:38.068472 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 16:09:38.069864 8444 catalog_manager.cc:4841] TS 2eff11aae3b941c390799581030ae587 (127.20.121.129:38631): tablet 4075d99f9b6d41ad9e690d4e6f187ef2 (table test-table [id=4087d2e864f34b95a8dc0758de0e747a]) successfully deleted
I20260929 16:09:38.070139 8638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:38.073750 8946 ts_tablet_manager.cc:1918] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:38.074034 8946 log.cc:1196] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-2/wal/wals/4075d99f9b6d41ad9e690d4e6f187ef2
I20260929 16:09:38.074729 8946 ts_tablet_manager.cc:1939] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 90892553dc304caebcd8506b9724f92d: Deleting consensus metadata
I20260929 16:09:38.075269 8947 ts_tablet_manager.cc:1918] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 16:09:38.075443 8947 log.cc:1196] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: Deleting WAL directory at /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790697841464038-20966-0/minicluster-data/ts-1/wal/wals/4075d99f9b6d41ad9e690d4e6f187ef2
I20260929 16:09:38.075968 8947 ts_tablet_manager.cc:1939] T 4075d99f9b6d41ad9e690d4e6f187ef2 P 52ced56cac074831964d4a9f07576ef0: Deleting consensus metadata
I20260929 16:09:38.076843 8443 catalog_manager.cc:4841] TS 90892553dc304caebcd8506b9724f92d (127.20.121.131:36867): tablet 4075d99f9b6d41ad9e690d4e6f187ef2 (table test-table [id=4087d2e864f34b95a8dc0758de0e747a]) successfully deleted
I20260929 16:09:38.077718 8441 catalog_manager.cc:4841] TS 52ced56cac074831964d4a9f07576ef0 (127.20.121.130:44911): tablet 4075d99f9b6d41ad9e690d4e6f187ef2 (table test-table [id=4087d2e864f34b95a8dc0758de0e747a]) successfully deleted
I20260929 16:09:38.175037 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 8516
I20260929 16:09:38.193915 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 8647
I20260929 16:09:38.202123 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:38.195141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55862 (local address 127.20.121.130:44911)
0929 16:09:38.195425 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:38.195431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:38.195448 (+ 17us) server_negotiation.cc:407] Connection header received
0929 16:09:38.195513 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:38.195521 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:38.195639 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:38.195737 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:38.196115 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:38.197259 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:38.198573 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:38.199216 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:38.199346 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:38.200512 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:38.200524 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:38.200544 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:38.201291 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:38.201368 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:38.201392 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:38.201498 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:38.201637 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 16:09:38.201749 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260929 16:09:38.202893 8769 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:38.293104 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 8647
I20260929 16:09:38.310235 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 8777
I20260929 16:09:38.317494 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:38.311130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.20.121.131:36867)
0929 16:09:38.311322 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:38.311329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:38.311369 (+ 40us) server_negotiation.cc:407] Connection header received
0929 16:09:38.311591 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:38.311595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:38.311682 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:38.311797 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:38.312209 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:38.312979 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:38.314344 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:38.314761 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:38.314920 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:38.316081 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:38.316090 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:38.316110 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:38.316699 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:38.316751 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:38.316771 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:38.316913 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:38.317047 (+ 134us) server_negotiation.cc:299] Negotiation successful
0929 16:09:38.317172 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 16:09:38.318377 8899 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:38.408749 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 8777
I20260929 16:09:38.425560 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 8426
I20260929 16:09:38.426951 8487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:38.538561 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 8426
2026-09-29T16:09:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2899 ms)
[----------] 6 tests from EncryptionPolicyTest (23594 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:38 dist-test-slave-x07d krb5kdc[8955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:38 dist-test-slave-x07d krb5kdc[8955](info): set up 2 sockets
Sep 29 16:09:38 dist-test-slave-x07d krb5kdc[8955](info): commencing operation
krb5kdc: starting...
W20260929 16:09:39.734548 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.006s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:39 dist-test-slave-x07d krb5kdc[8955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:09:40.006455 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:40.366744 8968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:40.367125 8968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:40.367290 8968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:40.374950 8968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:40.375141 8968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:40.375262 8968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:40.375376 8968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:40.375452 8968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:40.385865 8968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42715
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:40.387295 8968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:40.388648 8968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:40.402570 8973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:40.403402 8974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:40.403558 8976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:40.404968 8968 server_base.cc:1034] running on GCE node
I20260929 16:09:40.405876 8968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:40.407632 8968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:40.408882 8968 hybrid_clock.cc:648] HybridClock initialized: now 1790698180408847 us; error 35 us; skew 500 ppm
I20260929 16:09:40.409325 8968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:40 dist-test-slave-x07d krb5kdc[8955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698180, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:40.417594 8968 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:09:40.419446 8968 webserver.cc:460] Webserver started at http://127.0.0.1:34227/ using document root <none> and password file <none>
I20260929 16:09:40.420238 8968 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:40.420400 8968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:40.420742 8968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:40.424153 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "b2862402d110467587bd7f4e92fb6869"
format_stamp: "Formatted at 2026-09-29 16:09:40 on dist-test-slave-x07d"
server_key: "dead69ad2832e0d4b642e791a48d729a"
server_key_iv: "214d777f66a41fa400a6639bceecef81"
server_key_version: "encryptionkey@0"
I20260929 16:09:40.425038 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "b2862402d110467587bd7f4e92fb6869"
format_stamp: "Formatted at 2026-09-29 16:09:40 on dist-test-slave-x07d"
server_key: "dead69ad2832e0d4b642e791a48d729a"
server_key_iv: "214d777f66a41fa400a6639bceecef81"
server_key_version: "encryptionkey@0"
I20260929 16:09:40.430826 8968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 16:09:40.435050 8983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:40.436609 8968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 16:09:40.436908 8968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "b2862402d110467587bd7f4e92fb6869"
format_stamp: "Formatted at 2026-09-29 16:09:40 on dist-test-slave-x07d"
server_key: "dead69ad2832e0d4b642e791a48d729a"
server_key_iv: "214d777f66a41fa400a6639bceecef81"
server_key_version: "encryptionkey@0"
I20260929 16:09:40.437147 8968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:40.456562 8968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:40.461534 8968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:40.461855 8968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:40.482633 8968 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42715
I20260929 16:09:40.482679 9034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42715 every 8 connection(s)
I20260929 16:09:40.484411 8968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:40.485090 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 8968
I20260929 16:09:40.485349 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:40.486336 20966 external_mini_cluster.cc:1428] Setting key f48743870218cafe9c68cdbb8ea758b0
I20260929 16:09:40.489517 9035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 16:09:40 dist-test-slave-x07d krb5kdc[8955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 16:09:40.503794 9035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869: Bootstrap starting.
I20260929 16:09:40.508195 9035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:40.509629 9035 log.cc:824] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:40.513379 9035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869: No bootstrap required, opened a new log
I20260929 16:09:40.514374 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:40.488578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46093 (local address 127.0.0.1:42715)
0929 16:09:40.489583 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:40.489601 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:40.489653 (+ 52us) server_negotiation.cc:407] Connection header received
0929 16:09:40.491980 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:40.492011 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:40.492564 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:40.493079 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:40.494839 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:40.497019 (+ 2180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:40.498537 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:40.499687 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:40.499889 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:40.504811 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:40.504823 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:40.504843 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:40.504897 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:40.509213 (+ 4316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:40.510268 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:40.510282 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:40.510294 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:40.510515 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:40.511049 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:40.511052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:40.511054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:40.511790 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:40.512122 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:40.512527 (+ 405us) server_negotiation.cc:299] Negotiation successful
0929 16:09:40.512873 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":519,"threads_started":1}
I20260929 16:09:40.519042 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:09:40.522502 9035 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2862402d110467587bd7f4e92fb6869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } }
I20260929 16:09:40.523003 9035 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:40.523129 9035 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2862402d110467587bd7f4e92fb6869, State: Initialized, Role: FOLLOWER
I20260929 16:09:40.523772 9035 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2862402d110467587bd7f4e92fb6869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } }
I20260929 16:09:40.524040 9035 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:40.524202 9035 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:40.524462 9035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:40.527288 9035 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2862402d110467587bd7f4e92fb6869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } }
I20260929 16:09:40.528015 9035 leader_election.cc:302] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2862402d110467587bd7f4e92fb6869; no voters:
I20260929 16:09:40.528867 9035 leader_election.cc:288] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:40.529001 9045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:40.531931 9035 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:40.537498 9045 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [term 1 LEADER]: Becoming Leader. State: Replica: b2862402d110467587bd7f4e92fb6869, State: Running, Role: LEADER
I20260929 16:09:40.538383 9045 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2862402d110467587bd7f4e92fb6869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } }
I20260929 16:09:40.543515 9047 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2862402d110467587bd7f4e92fb6869. Latest consensus state: current_term: 1 leader_uuid: "b2862402d110467587bd7f4e92fb6869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2862402d110467587bd7f4e92fb6869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } } }
I20260929 16:09:40.543908 9047 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:40.544668 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:40.524554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35493 (local address 127.0.0.1:42715)
0929 16:09:40.524794 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:40.524804 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:40.525417 (+ 613us) server_negotiation.cc:407] Connection header received
0929 16:09:40.525602 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:40.525609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:40.525695 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:40.525821 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:40.527461 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:40.528672 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:40.530455 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:40.531109 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:40.531256 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:40.533912 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:40.533921 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:40.533926 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:40.533973 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:40.539574 (+ 5601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:40.540918 (+ 1344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:40.540923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:40.540926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:40.541128 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:40.541755 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:40.541758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:40.541761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:40.542343 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:40.543908 (+ 1565us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:40.544102 (+ 194us) server_negotiation.cc:299] Negotiation successful
0929 16:09:40.544296 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 16:09:40.545310 9046 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2862402d110467587bd7f4e92fb6869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2862402d110467587bd7f4e92fb6869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42715 } } }
I20260929 16:09:40.545629 9046 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:40.546983 9059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:40.551519 9059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:40.561118 9059 catalog_manager.cc:1347] Generated new cluster ID: 0ceba45e1d884f8284e5127fdea9a856
I20260929 16:09:40.561255 9059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:40.575583 9059 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:40.577543 9059 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:40.589461 9059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2862402d110467587bd7f4e92fb6869: Generated new TSK 0
I20260929 16:09:40.590502 9059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:40.606318 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 8968
I20260929 16:09:40.608322 9030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:40.701857 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 8968
W20260929 16:09:40.719915 8953 connection.cc:535] client connection to 127.0.0.1:42715 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42715): Connection reset by peer (error 104)
2026-09-29T16:09:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2157 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:40 dist-test-slave-x07d krb5kdc[9067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:40 dist-test-slave-x07d krb5kdc[9067](info): set up 2 sockets
Sep 29 16:09:40 dist-test-slave-x07d krb5kdc[9067](info): commencing operation
krb5kdc: starting...
W20260929 16:09:41.822502 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:41 dist-test-slave-x07d krb5kdc[9067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:09:42.091604 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:42.446131 9080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:42.446491 9080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:42.446681 9080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:42.454456 9080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:42.454607 9080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:42.454749 9080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:42.455041 9080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:42.455148 9080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:42.466109 9080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42609
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:42.467561 9080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:42.468993 9080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:42.482277 9086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:42.482326 9085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:42.482360 9088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:42.483724 9080 server_base.cc:1034] running on GCE node
I20260929 16:09:42.484668 9080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:42.486371 9080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:42.487607 9080 hybrid_clock.cc:648] HybridClock initialized: now 1790698182487537 us; error 64 us; skew 500 ppm
I20260929 16:09:42.487965 9080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:42 dist-test-slave-x07d krb5kdc[9067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698182, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:42.492019 9080 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:09:42.493743 9080 webserver.cc:460] Webserver started at http://127.0.0.1:34747/ using document root <none> and password file <none>
I20260929 16:09:42.494568 9080 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:42.494725 9080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:42.495059 9080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:42.499234 9080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "39f43bdb247248ec9b60fa1adce149c2"
format_stamp: "Formatted at 2026-09-29 16:09:42 on dist-test-slave-x07d"
server_key: "3427ce4a5061c4fb3908bc5e18701426"
server_key_iv: "57a5cb9999b3fd82df093c57bee734a9"
server_key_version: "encryptionkey@0"
I20260929 16:09:42.500180 9080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "39f43bdb247248ec9b60fa1adce149c2"
format_stamp: "Formatted at 2026-09-29 16:09:42 on dist-test-slave-x07d"
server_key: "3427ce4a5061c4fb3908bc5e18701426"
server_key_iv: "57a5cb9999b3fd82df093c57bee734a9"
server_key_version: "encryptionkey@0"
I20260929 16:09:42.505921 9080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 16:09:42.509897 9095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:42.511480 9080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260929 16:09:42.511708 9080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "39f43bdb247248ec9b60fa1adce149c2"
format_stamp: "Formatted at 2026-09-29 16:09:42 on dist-test-slave-x07d"
server_key: "3427ce4a5061c4fb3908bc5e18701426"
server_key_iv: "57a5cb9999b3fd82df093c57bee734a9"
server_key_version: "encryptionkey@0"
I20260929 16:09:42.511945 9080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:42.528697 9080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:42.534085 9080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:42.534437 9080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:42.554217 9080 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42609
I20260929 16:09:42.554220 9146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42609 every 8 connection(s)
I20260929 16:09:42.555804 9080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:42.560245 9147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:42.560293 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 9080
I20260929 16:09:42.560521 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:42.560999 20966 external_mini_cluster.cc:1428] Setting key 1e0de4607a4beed113229674325a3e0c
I20260929 16:09:42.576997 9147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2: Bootstrap starting.
I20260929 16:09:42.577397 9150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:42.562914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38583 (local address 127.0.0.1:42609)
0929 16:09:42.563870 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:42.563887 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:42.563940 (+ 53us) server_negotiation.cc:407] Connection header received
0929 16:09:42.565897 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:42.565936 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:42.566672 (+ 736us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:42.567202 (+ 530us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:42.567827 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:42.569909 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:42.571482 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:42.572641 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:42.572852 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:42.574349 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:42.574360 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:42.574385 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:42.575151 (+ 766us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:42.575256 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:42.575296 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:42.575487 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:42.575784 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 16:09:42.575983 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":387,"threads_started":1}
I20260929 16:09:42.581041 9147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:42.582564 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:09:42.582530 9147 log.cc:824] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:42.586520 9147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2: No bootstrap required, opened a new log
I20260929 16:09:42.595928 9150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:42.588313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44283 (local address 127.0.0.1:42609)
0929 16:09:42.588534 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:42.588545 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:42.588963 (+ 418us) server_negotiation.cc:407] Connection header received
0929 16:09:42.589141 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:42.589148 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:42.589239 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:42.589366 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:42.589962 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:42.591182 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:42.592651 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:42.593382 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:42.593518 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:42.594965 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:42.594972 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:42.594977 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:42.595008 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:42.595081 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:42.595104 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:42.595248 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:42.595482 (+ 234us) server_negotiation.cc:299] Negotiation successful
0929 16:09:42.595551 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:09:42.597263 9147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f43bdb247248ec9b60fa1adce149c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42609 } }
I20260929 16:09:42.597793 9147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:42.597922 9147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f43bdb247248ec9b60fa1adce149c2, State: Initialized, Role: FOLLOWER
I20260929 16:09:42.598631 9147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f43bdb247248ec9b60fa1adce149c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42609 } }
I20260929 16:09:42.598891 9147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:42.599052 9147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:42.599232 9147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:42.601678 9147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f43bdb247248ec9b60fa1adce149c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42609 } }
I20260929 16:09:42.602355 9147 leader_election.cc:302] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39f43bdb247248ec9b60fa1adce149c2; no voters:
I20260929 16:09:42.603099 9147 leader_election.cc:288] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:42.603295 9157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:42.604686 9157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [term 1 LEADER]: Becoming Leader. State: Replica: 39f43bdb247248ec9b60fa1adce149c2, State: Running, Role: LEADER
I20260929 16:09:42.605363 9157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f43bdb247248ec9b60fa1adce149c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42609 } }
I20260929 16:09:42.605522 9147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:42.608824 9159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39f43bdb247248ec9b60fa1adce149c2. Latest consensus state: current_term: 1 leader_uuid: "39f43bdb247248ec9b60fa1adce149c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f43bdb247248ec9b60fa1adce149c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42609 } } }
I20260929 16:09:42.609313 9159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:42.608983 9158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39f43bdb247248ec9b60fa1adce149c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f43bdb247248ec9b60fa1adce149c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42609 } } }
I20260929 16:09:42.609453 9158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:42.615083 9166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:42.621729 9166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:42.631776 9166 catalog_manager.cc:1347] Generated new cluster ID: 4c0e268141744ecb8a3c279550e0828b
I20260929 16:09:42.631927 9166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:42.650104 9166 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:42.651576 9166 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:42.660542 9166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39f43bdb247248ec9b60fa1adce149c2: Generated new TSK 0
I20260929 16:09:42.661619 9166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:42.671799 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9080
W20260929 16:09:42.673112 9142 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38583
W20260929 16:09:42.674206 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9080: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:09:42.674425 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 9080
W20260929 16:09:42.693284 9065 connection.cc:535] client connection to 127.0.0.1:42609 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42609): Connection reset by peer (error 104)
2026-09-29T16:09:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1973 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:42 dist-test-slave-x07d krb5kdc[9178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:42 dist-test-slave-x07d krb5kdc[9178](info): set up 2 sockets
Sep 29 16:09:42 dist-test-slave-x07d krb5kdc[9178](info): commencing operation
krb5kdc: starting...
W20260929 16:09:43.810505 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:43 dist-test-slave-x07d krb5kdc[9178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:09:44.089625 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33043
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:44.444383 9191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:44.444906 9191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:44.445083 9191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:44.453450 9191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:44.453673 9191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:44.453789 9191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:44.453907 9191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:44.454123 9191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:44.466028 9191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33043
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:44.467542 9191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:44.468983 9191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:44.481422 9196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:44.481463 9197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:44.481695 9199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:44.482811 9191 server_base.cc:1034] running on GCE node
I20260929 16:09:44.483671 9191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:44.485229 9191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:44.486495 9191 hybrid_clock.cc:648] HybridClock initialized: now 1790698184486427 us; error 62 us; skew 500 ppm
I20260929 16:09:44.486903 9191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:44 dist-test-slave-x07d krb5kdc[9178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698184, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:44.490420 9191 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:09:44.492061 9191 webserver.cc:460] Webserver started at http://127.0.0.1:35345/ using document root <none> and password file <none>
I20260929 16:09:44.492799 9191 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:44.492956 9191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:44.493270 9191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:44.496568 9191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "2e4df24fb331440da1730809b5069029"
format_stamp: "Formatted at 2026-09-29 16:09:44 on dist-test-slave-x07d"
server_key: "a1f2c5dd29337904a2fd5d115270ad4b"
server_key_iv: "eaa5335e2432676ba793143b6c38a37d"
server_key_version: "encryptionkey@0"
I20260929 16:09:44.497430 9191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "2e4df24fb331440da1730809b5069029"
format_stamp: "Formatted at 2026-09-29 16:09:44 on dist-test-slave-x07d"
server_key: "a1f2c5dd29337904a2fd5d115270ad4b"
server_key_iv: "eaa5335e2432676ba793143b6c38a37d"
server_key_version: "encryptionkey@0"
I20260929 16:09:44.503142 9191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 16:09:44.507162 9206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:44.508677 9191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 16:09:44.508888 9191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "2e4df24fb331440da1730809b5069029"
format_stamp: "Formatted at 2026-09-29 16:09:44 on dist-test-slave-x07d"
server_key: "a1f2c5dd29337904a2fd5d115270ad4b"
server_key_iv: "eaa5335e2432676ba793143b6c38a37d"
server_key_version: "encryptionkey@0"
I20260929 16:09:44.509114 9191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:44.515282 9191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:44.520206 9191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:44.520546 9191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:44.541776 9191 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33043
I20260929 16:09:44.541805 9257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33043 every 8 connection(s)
I20260929 16:09:44.543601 9191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:44.547030 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 9191
I20260929 16:09:44.547214 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:44.547757 20966 external_mini_cluster.cc:1428] Setting key 8bd8eff70319532e88d7773b785a8761
I20260929 16:09:44.548224 9258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:44.557040 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:44.549789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41503 (local address 127.0.0.1:33043)
0929 16:09:44.550698 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:44.550710 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:44.550752 (+ 42us) server_negotiation.cc:407] Connection header received
0929 16:09:44.553132 (+ 2380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:44.553156 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:44.553553 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:44.553929 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:44.555200 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:44.555207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:44.555221 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:44.555237 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:44.555324 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:44.555349 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:44.555537 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:44.555674 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 16:09:44.556002 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":468,"threads_started":1}
I20260929 16:09:44.561751 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:09:44.563700 9258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029: Bootstrap starting.
I20260929 16:09:44.568320 9258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:44.569850 9258 log.cc:824] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:44.570196 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:44.567602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58437 (local address 127.0.0.1:33043)
0929 16:09:44.567796 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:44.567802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:44.568065 (+ 263us) server_negotiation.cc:407] Connection header received
0929 16:09:44.568236 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:44.568240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:44.568293 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:44.568388 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:44.569360 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:44.569364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:44.569367 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:44.569381 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:44.569424 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:44.569439 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:44.569536 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:44.569692 (+ 156us) server_negotiation.cc:299] Negotiation successful
0929 16:09:44.569740 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 16:09:44.573184 9258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029: No bootstrap required, opened a new log
I20260929 16:09:44.581875 9258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4df24fb331440da1730809b5069029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260929 16:09:44.582394 9258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:44.582512 9258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e4df24fb331440da1730809b5069029, State: Initialized, Role: FOLLOWER
I20260929 16:09:44.583158 9258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4df24fb331440da1730809b5069029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260929 16:09:44.583428 9258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:44.583583 9258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:44.583822 9258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:44.586592 9258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4df24fb331440da1730809b5069029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260929 16:09:44.587229 9258 leader_election.cc:302] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e4df24fb331440da1730809b5069029; no voters:
I20260929 16:09:44.587919 9258 leader_election.cc:288] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:44.588085 9268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:44.588822 9268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [term 1 LEADER]: Becoming Leader. State: Replica: 2e4df24fb331440da1730809b5069029, State: Running, Role: LEADER
I20260929 16:09:44.589421 9268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4df24fb331440da1730809b5069029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } }
I20260929 16:09:44.590282 9258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:44.592646 9269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e4df24fb331440da1730809b5069029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4df24fb331440da1730809b5069029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } } }
I20260929 16:09:44.593142 9269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:44.592773 9270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e4df24fb331440da1730809b5069029. Latest consensus state: current_term: 1 leader_uuid: "2e4df24fb331440da1730809b5069029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e4df24fb331440da1730809b5069029" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33043 } } }
I20260929 16:09:44.593294 9270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:44.600297 9274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:44.605655 9274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:44.620817 9274 catalog_manager.cc:1347] Generated new cluster ID: 022660fc71e04b63957cf7235b30ac4e
I20260929 16:09:44.620963 9274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:44.639518 9274 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:44.640004 9274 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:44.654436 9274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e4df24fb331440da1730809b5069029: Generated new TSK 0
I20260929 16:09:44.655682 9274 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:44.669700 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9191
W20260929 16:09:44.670917 9253 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41503
W20260929 16:09:44.671706 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9191: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:09:44.671860 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 9191
2026-09-29T16:09:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1995 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:44 dist-test-slave-x07d krb5kdc[9289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:44 dist-test-slave-x07d krb5kdc[9289](info): set up 2 sockets
Sep 29 16:09:44 dist-test-slave-x07d krb5kdc[9289](info): commencing operation
krb5kdc: starting...
W20260929 16:09:45.806536 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:45 dist-test-slave-x07d krb5kdc[9289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:09:46.085979 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33621
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:46.439391 9302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:46.439822 9302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:46.439997 9302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:46.447710 9302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:46.447914 9302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:46.448068 9302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:46.448185 9302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:46.448259 9302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:46.458778 9302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:46.460290 9302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:46.461694 9302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:46.475364 9308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:46.475345 9307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:46.475394 9310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:46.476841 9302 server_base.cc:1034] running on GCE node
I20260929 16:09:46.477847 9302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:46.479664 9302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:46.480969 9302 hybrid_clock.cc:648] HybridClock initialized: now 1790698186480902 us; error 80 us; skew 500 ppm
I20260929 16:09:46.481454 9302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:46 dist-test-slave-x07d krb5kdc[9289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698186, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:46.492515 9302 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:09:46.494524 9302 webserver.cc:460] Webserver started at http://127.0.0.1:38487/ using document root <none> and password file <none>
I20260929 16:09:46.495270 9302 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:46.495388 9302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:46.495654 9302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:46.499032 9302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "545f61a1f1324af48315d543160647a8"
format_stamp: "Formatted at 2026-09-29 16:09:46 on dist-test-slave-x07d"
server_key: "4c28753b23e554cd695b95ec0e4b849b"
server_key_iv: "660ea0336fe9bc6dabcf3d552dbcd608"
server_key_version: "encryptionkey@0"
I20260929 16:09:46.499864 9302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "545f61a1f1324af48315d543160647a8"
format_stamp: "Formatted at 2026-09-29 16:09:46 on dist-test-slave-x07d"
server_key: "4c28753b23e554cd695b95ec0e4b849b"
server_key_iv: "660ea0336fe9bc6dabcf3d552dbcd608"
server_key_version: "encryptionkey@0"
I20260929 16:09:46.505849 9302 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260929 16:09:46.510166 9317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:46.511854 9302 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 16:09:46.512089 9302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "545f61a1f1324af48315d543160647a8"
format_stamp: "Formatted at 2026-09-29 16:09:46 on dist-test-slave-x07d"
server_key: "4c28753b23e554cd695b95ec0e4b849b"
server_key_iv: "660ea0336fe9bc6dabcf3d552dbcd608"
server_key_version: "encryptionkey@0"
I20260929 16:09:46.512316 9302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:46.534729 9302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:46.539932 9302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:46.540263 9302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:46.561028 9302 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33621
I20260929 16:09:46.561060 9368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33621 every 8 connection(s)
I20260929 16:09:46.562588 9302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:46.564006 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 9302
I20260929 16:09:46.564208 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:46.564786 20966 external_mini_cluster.cc:1428] Setting key 66025f1109cf7ee74371bfc62461aeb1
I20260929 16:09:46.567953 9369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 16:09:46 dist-test-slave-x07d krb5kdc[9289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 16:09:46.580780 9369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8: Bootstrap starting.
I20260929 16:09:46.585043 9369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:46.586548 9369 log.cc:824] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:46.589320 9372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:46.566826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52891 (local address 127.0.0.1:33621)
0929 16:09:46.567717 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:46.567729 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:46.567773 (+ 44us) server_negotiation.cc:407] Connection header received
0929 16:09:46.569603 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:46.569627 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:46.570134 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:46.570555 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:46.572149 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:46.573610 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:46.575142 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:46.575889 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:46.576114 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:46.580831 (+ 4717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:46.580840 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:46.580885 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:46.580951 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:46.584454 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:46.585504 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:46.585512 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:46.585518 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:46.585700 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:46.586177 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:46.586180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:46.586183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:46.586857 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:46.587093 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:46.587749 (+ 656us) server_negotiation.cc:299] Negotiation successful
0929 16:09:46.588251 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":380,"threads_started":1}
I20260929 16:09:46.590140 9369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8: No bootstrap required, opened a new log
I20260929 16:09:46.593659 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:09:46.599936 9369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545f61a1f1324af48315d543160647a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33621 } }
I20260929 16:09:46.600409 9369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:46.600540 9369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545f61a1f1324af48315d543160647a8, State: Initialized, Role: FOLLOWER
I20260929 16:09:46.601183 9369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545f61a1f1324af48315d543160647a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33621 } }
I20260929 16:09:46.601531 9369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:46.601696 9369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:46.601914 9369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:46.605185 9369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545f61a1f1324af48315d543160647a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33621 } }
I20260929 16:09:46.605818 9369 leader_election.cc:302] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545f61a1f1324af48315d543160647a8; no voters:
I20260929 16:09:46.606566 9369 leader_election.cc:288] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:46.606746 9379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:46.607542 9379 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [term 1 LEADER]: Becoming Leader. State: Replica: 545f61a1f1324af48315d543160647a8, State: Running, Role: LEADER
I20260929 16:09:46.608345 9379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545f61a1f1324af48315d543160647a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33621 } }
I20260929 16:09:46.610425 9369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:46.611584 9381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "545f61a1f1324af48315d543160647a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545f61a1f1324af48315d543160647a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33621 } } }
I20260929 16:09:46.612095 9381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:46.613502 9380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545f61a1f1324af48315d543160647a8. Latest consensus state: current_term: 1 leader_uuid: "545f61a1f1324af48315d543160647a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545f61a1f1324af48315d543160647a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33621 } } }
I20260929 16:09:46.613987 9380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:46.618774 9372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:46.598472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49657 (local address 127.0.0.1:33621)
0929 16:09:46.598700 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:46.598707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:46.599177 (+ 470us) server_negotiation.cc:407] Connection header received
0929 16:09:46.599336 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:46.599341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:46.599449 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:46.599583 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:46.601338 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:46.602621 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:46.603947 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:46.604714 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:46.604869 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:46.608216 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:46.608228 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:46.608237 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:46.608296 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:46.613067 (+ 4771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:46.614384 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:46.614393 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:46.614399 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:46.614647 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:46.615230 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:46.615238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:46.615244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:46.615749 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:46.617791 (+ 2042us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:46.617998 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 16:09:46.618363 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 16:09:46.619686 9384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:46.626829 9384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:46.636643 9384 catalog_manager.cc:1347] Generated new cluster ID: 2a90d58416854279bb38860197ca304b
I20260929 16:09:46.636792 9384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:46.647445 9384 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:46.649160 9384 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:46.663897 9384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 545f61a1f1324af48315d543160647a8: Generated new TSK 0
I20260929 16:09:46.665217 9384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:46.678681 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9302
I20260929 16:09:46.680409 9364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:46.776288 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 9302
2026-09-29T16:09:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2105 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:46 dist-test-slave-x07d krb5kdc[9401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:46 dist-test-slave-x07d krb5kdc[9401](info): set up 2 sockets
Sep 29 16:09:46 dist-test-slave-x07d krb5kdc[9401](info): commencing operation
krb5kdc: starting...
W20260929 16:09:47.914662 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.008s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:48 dist-test-slave-x07d krb5kdc[9401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:09:48.210850 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36665
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:48.556797 9414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:48.557145 9414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:48.557243 9414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:48.565310 9414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:48.565475 9414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:48.565572 9414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:48.565661 9414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:48.565708 9414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:48.576138 9414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:48.577543 9414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:48.579339 9414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:48.592391 9419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:48.592458 9422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:48.592479 9420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:48.592983 9414 server_base.cc:1034] running on GCE node
I20260929 16:09:48.593997 9414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:48.595650 9414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:48.596930 9414 hybrid_clock.cc:648] HybridClock initialized: now 1790698188596884 us; error 63 us; skew 500 ppm
I20260929 16:09:48.597342 9414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:48 dist-test-slave-x07d krb5kdc[9401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698188, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:48.601032 9414 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:09:48.602751 9414 webserver.cc:460] Webserver started at http://127.0.0.1:39519/ using document root <none> and password file <none>
I20260929 16:09:48.603462 9414 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:48.603574 9414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:48.603852 9414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:48.607189 9414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "9c7876e515554a988804b27a81b32ca1"
format_stamp: "Formatted at 2026-09-29 16:09:48 on dist-test-slave-x07d"
server_key: "5c86faf7a0cf0a7b42355ee1c5091aed"
server_key_iv: "99379ccda9f97bd440fca7fbd28ed7a4"
server_key_version: "encryptionkey@0"
I20260929 16:09:48.608093 9414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "9c7876e515554a988804b27a81b32ca1"
format_stamp: "Formatted at 2026-09-29 16:09:48 on dist-test-slave-x07d"
server_key: "5c86faf7a0cf0a7b42355ee1c5091aed"
server_key_iv: "99379ccda9f97bd440fca7fbd28ed7a4"
server_key_version: "encryptionkey@0"
I20260929 16:09:48.614169 9414 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260929 16:09:48.618387 9429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:48.620023 9414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:09:48.620208 9414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "9c7876e515554a988804b27a81b32ca1"
format_stamp: "Formatted at 2026-09-29 16:09:48 on dist-test-slave-x07d"
server_key: "5c86faf7a0cf0a7b42355ee1c5091aed"
server_key_iv: "99379ccda9f97bd440fca7fbd28ed7a4"
server_key_version: "encryptionkey@0"
I20260929 16:09:48.620430 9414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:48.637964 9414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:48.642688 9414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:48.642947 9414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:48.662519 9414 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36665
I20260929 16:09:48.662571 9480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36665 every 8 connection(s)
I20260929 16:09:48.664024 9414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:48.667702 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 9414
I20260929 16:09:48.667923 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:48.668345 9481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:48.668458 20966 external_mini_cluster.cc:1428] Setting key 76acd0dd8ae52051681f74cbef2330c7
I20260929 16:09:48.682507 9481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1: Bootstrap starting.
I20260929 16:09:48.682618 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:48.670452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43109 (local address 127.0.0.1:36665)
0929 16:09:48.671245 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:48.671257 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:48.671299 (+ 42us) server_negotiation.cc:407] Connection header received
0929 16:09:48.673356 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:48.673381 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:48.673832 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:48.674387 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:48.674987 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:48.676331 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:48.677782 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:48.678555 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:48.678761 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:48.680076 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:48.680083 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:48.680100 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:48.680693 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:48.680780 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:48.680806 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:48.680958 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:48.681206 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 16:09:48.681482 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":362,"threads_started":1}
I20260929 16:09:48.686399 9481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:48.686725 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:09:48.687578 9481 log.cc:824] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:48.690666 9481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1: No bootstrap required, opened a new log
I20260929 16:09:48.697963 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:48.691239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54029 (local address 127.0.0.1:36665)
0929 16:09:48.691437 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:48.691444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:48.691854 (+ 410us) server_negotiation.cc:407] Connection header received
0929 16:09:48.692025 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:48.692031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:48.692108 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:48.692223 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:48.692707 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:48.693845 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:48.695175 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:48.695884 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:48.696042 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:48.697158 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:48.697164 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:48.697170 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:48.697194 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:48.697247 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:48.697267 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:48.697370 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:48.697497 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 16:09:48.697597 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 16:09:48.700850 9481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7876e515554a988804b27a81b32ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } }
I20260929 16:09:48.701212 9481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:48.701290 9481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7876e515554a988804b27a81b32ca1, State: Initialized, Role: FOLLOWER
I20260929 16:09:48.701795 9481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7876e515554a988804b27a81b32ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } }
I20260929 16:09:48.702128 9481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:48.702239 9481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:48.702363 9481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:48.705086 9481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7876e515554a988804b27a81b32ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } }
I20260929 16:09:48.705583 9481 leader_election.cc:302] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c7876e515554a988804b27a81b32ca1; no voters:
I20260929 16:09:48.706171 9481 leader_election.cc:288] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:48.706302 9491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:48.707067 9491 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 9c7876e515554a988804b27a81b32ca1, State: Running, Role: LEADER
I20260929 16:09:48.707648 9491 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7876e515554a988804b27a81b32ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } }
I20260929 16:09:48.708384 9481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:48.710616 9492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c7876e515554a988804b27a81b32ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7876e515554a988804b27a81b32ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } } }
I20260929 16:09:48.711086 9492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:48.711431 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c7876e515554a988804b27a81b32ca1. Latest consensus state: current_term: 1 leader_uuid: "9c7876e515554a988804b27a81b32ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7876e515554a988804b27a81b32ca1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36665 } } }
I20260929 16:09:48.711740 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:48.715374 9497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:48.721166 9497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:48.730767 9497 catalog_manager.cc:1347] Generated new cluster ID: 9c16632dc74c4d05b25b7b9e74376a7d
I20260929 16:09:48.730901 9497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:48.744485 9497 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:48.746526 9497 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:48.763908 9497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c7876e515554a988804b27a81b32ca1: Generated new TSK 0
I20260929 16:09:48.765106 9497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:48.777285 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9414
W20260929 16:09:48.778641 9476 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43109
W20260929 16:09:48.779574 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9414: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:09:48.779754 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 9414
2026-09-29T16:09:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2002 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:48 dist-test-slave-x07d krb5kdc[9512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:48 dist-test-slave-x07d krb5kdc[9512](info): set up 2 sockets
Sep 29 16:09:48 dist-test-slave-x07d krb5kdc[9512](info): commencing operation
krb5kdc: starting...
W20260929 16:09:49.990525 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.005s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:50 dist-test-slave-x07d krb5kdc[9512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:09:50.274472 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45577
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:50.632862 9525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:50.633184 9525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:50.633279 9525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:50.641243 9525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:50.641390 9525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:50.641441 9525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:50.641510 9525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:50.641554 9525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:50.652184 9525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:50.653578 9525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:50.655026 9525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:50.669082 9530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:50.669308 9531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:50.669308 9533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:50.670473 9525 server_base.cc:1034] running on GCE node
I20260929 16:09:50.671368 9525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:50.673017 9525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:50.674275 9525 hybrid_clock.cc:648] HybridClock initialized: now 1790698190674223 us; error 37 us; skew 500 ppm
I20260929 16:09:50.674907 9525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:50 dist-test-slave-x07d krb5kdc[9512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698190, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:50.679165 9525 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:09:50.681334 9525 webserver.cc:460] Webserver started at http://127.0.0.1:38967/ using document root <none> and password file <none>
I20260929 16:09:50.682286 9525 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:50.682466 9525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:50.682839 9525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:50.686458 9525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "2fbf3a41a9fa42688e0719946ddf16eb"
format_stamp: "Formatted at 2026-09-29 16:09:50 on dist-test-slave-x07d"
server_key: "f4a1adfafc2beb7181af4cd776e7bd2c"
server_key_iv: "2d3e69c9680a96599644e12a7c0628fa"
server_key_version: "encryptionkey@0"
I20260929 16:09:50.687451 9525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "2fbf3a41a9fa42688e0719946ddf16eb"
format_stamp: "Formatted at 2026-09-29 16:09:50 on dist-test-slave-x07d"
server_key: "f4a1adfafc2beb7181af4cd776e7bd2c"
server_key_iv: "2d3e69c9680a96599644e12a7c0628fa"
server_key_version: "encryptionkey@0"
I20260929 16:09:50.693813 9525 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260929 16:09:50.698264 9540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:50.700140 9525 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 16:09:50.700433 9525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "2fbf3a41a9fa42688e0719946ddf16eb"
format_stamp: "Formatted at 2026-09-29 16:09:50 on dist-test-slave-x07d"
server_key: "f4a1adfafc2beb7181af4cd776e7bd2c"
server_key_iv: "2d3e69c9680a96599644e12a7c0628fa"
server_key_version: "encryptionkey@0"
I20260929 16:09:50.700685 9525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:50.707739 9525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:50.713515 9525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:50.713995 9525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:50.736361 9525 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45577
I20260929 16:09:50.736393 9591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45577 every 8 connection(s)
I20260929 16:09:50.737885 9525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:50.742102 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 9525
I20260929 16:09:50.742195 9592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:50.742332 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:50.742865 20966 external_mini_cluster.cc:1428] Setting key de8b87d0d601c15bab8566fd5ccd9706
I20260929 16:09:50.751708 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:50.744842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60247 (local address 127.0.0.1:45577)
0929 16:09:50.745884 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:50.745898 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:50.745948 (+ 50us) server_negotiation.cc:407] Connection header received
0929 16:09:50.747735 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:50.747761 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:50.748172 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:50.748567 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:50.749803 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:50.749811 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:50.749828 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:50.749915 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:50.749983 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:50.750063 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:50.750287 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:50.750409 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 16:09:50.750649 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":490,"threads_started":1}
I20260929 16:09:50.756415 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:09:50.757958 9592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb: Bootstrap starting.
I20260929 16:09:50.762130 9592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:50.763724 9592 log.cc:824] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:50.764472 9595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:50.761637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53521 (local address 127.0.0.1:45577)
0929 16:09:50.761890 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:50.761897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:50.762269 (+ 372us) server_negotiation.cc:407] Connection header received
0929 16:09:50.762499 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:50.762503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:50.762568 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:50.762661 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:50.763698 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:50.763703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:50.763706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:50.763724 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:50.763766 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:50.763782 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:50.763942 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:50.764081 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 16:09:50.764134 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 16:09:50.768748 9592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb: No bootstrap required, opened a new log
I20260929 16:09:50.777216 9592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbf3a41a9fa42688e0719946ddf16eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45577 } }
I20260929 16:09:50.777663 9592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:50.777904 9592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fbf3a41a9fa42688e0719946ddf16eb, State: Initialized, Role: FOLLOWER
I20260929 16:09:50.778687 9592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbf3a41a9fa42688e0719946ddf16eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45577 } }
I20260929 16:09:50.778940 9592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:50.779050 9592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:50.779240 9592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:50.782191 9592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbf3a41a9fa42688e0719946ddf16eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45577 } }
I20260929 16:09:50.782753 9592 leader_election.cc:302] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fbf3a41a9fa42688e0719946ddf16eb; no voters:
I20260929 16:09:50.783329 9592 leader_election.cc:288] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:50.783483 9602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:50.784318 9602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [term 1 LEADER]: Becoming Leader. State: Replica: 2fbf3a41a9fa42688e0719946ddf16eb, State: Running, Role: LEADER
I20260929 16:09:50.784993 9602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbf3a41a9fa42688e0719946ddf16eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45577 } }
I20260929 16:09:50.785856 9592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:50.788093 9604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fbf3a41a9fa42688e0719946ddf16eb. Latest consensus state: current_term: 1 leader_uuid: "2fbf3a41a9fa42688e0719946ddf16eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbf3a41a9fa42688e0719946ddf16eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45577 } } }
I20260929 16:09:50.788446 9603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fbf3a41a9fa42688e0719946ddf16eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fbf3a41a9fa42688e0719946ddf16eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45577 } } }
I20260929 16:09:50.788771 9603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:50.788647 9604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:50.790714 9607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:50.796921 9607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:50.807175 9607 catalog_manager.cc:1347] Generated new cluster ID: 5623e7010f934a15a7e08789e0d925a2
I20260929 16:09:50.807361 9607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:50.827230 9607 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:50.827728 9607 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:50.835898 9607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fbf3a41a9fa42688e0719946ddf16eb: Generated new TSK 0
I20260929 16:09:50.836802 9607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:50.847641 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9525
W20260929 16:09:50.848798 9587 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60247
W20260929 16:09:50.849541 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9525: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:09:50.849677 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 9525
2026-09-29T16:09:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2070 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:50 dist-test-slave-x07d krb5kdc[9623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:50 dist-test-slave-x07d krb5kdc[9623](info): set up 2 sockets
Sep 29 16:09:50 dist-test-slave-x07d krb5kdc[9623](info): commencing operation
krb5kdc: starting...
W20260929 16:09:52.062672 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:52 dist-test-slave-x07d krb5kdc[9623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:09:52.350039 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42149
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:52.702476 9636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:52.702870 9636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:52.703039 9636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:52.710700 9636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:52.710878 9636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:52.710996 9636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:52.711102 9636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:52.711186 9636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:52.721683 9636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:52.723197 9636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:52.724651 9636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:52.737907 9641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:52.737795 9642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:52.738060 9644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:52.739017 9636 server_base.cc:1034] running on GCE node
I20260929 16:09:52.739897 9636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:52.741534 9636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:52.742805 9636 hybrid_clock.cc:648] HybridClock initialized: now 1790698192742731 us; error 68 us; skew 500 ppm
I20260929 16:09:52.743269 9636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:52 dist-test-slave-x07d krb5kdc[9623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698192, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:52.754117 9636 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:09:52.756031 9636 webserver.cc:460] Webserver started at http://127.0.0.1:33767/ using document root <none> and password file <none>
I20260929 16:09:52.756776 9636 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:52.756902 9636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:52.757189 9636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:52.760640 9636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "ea18bf2564394d96a90d35a82ed2cf5c"
format_stamp: "Formatted at 2026-09-29 16:09:52 on dist-test-slave-x07d"
server_key: "3f4270001e3e1032134b473d85555eff"
server_key_iv: "d8328b038b3a397ca22b25180ef1521a"
server_key_version: "encryptionkey@0"
I20260929 16:09:52.761480 9636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "ea18bf2564394d96a90d35a82ed2cf5c"
format_stamp: "Formatted at 2026-09-29 16:09:52 on dist-test-slave-x07d"
server_key: "3f4270001e3e1032134b473d85555eff"
server_key_iv: "d8328b038b3a397ca22b25180ef1521a"
server_key_version: "encryptionkey@0"
I20260929 16:09:52.768041 9636 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.001s
I20260929 16:09:52.772367 9651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:52.774179 9636 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260929 16:09:52.774416 9636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "ea18bf2564394d96a90d35a82ed2cf5c"
format_stamp: "Formatted at 2026-09-29 16:09:52 on dist-test-slave-x07d"
server_key: "3f4270001e3e1032134b473d85555eff"
server_key_iv: "d8328b038b3a397ca22b25180ef1521a"
server_key_version: "encryptionkey@0"
I20260929 16:09:52.774668 9636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:52.803665 9636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:52.808446 9636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:52.808766 9636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:52.829058 9636 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42149
I20260929 16:09:52.829094 9702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42149 every 8 connection(s)
I20260929 16:09:52.830598 9636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:52.834610 9703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:52.838404 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 9636
I20260929 16:09:52.838605 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:52.839249 20966 external_mini_cluster.cc:1428] Setting key 15685a2a34143a1839616d17af7f74d5
I20260929 16:09:52.848510 9703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c: Bootstrap starting.
I20260929 16:09:52.852923 9703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:52.854575 9703 log.cc:824] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c: Log is configured to *not* fsync() on all Append() calls
Sep 29 16:09:52 dist-test-slave-x07d krb5kdc[9623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 16:09:52.858434 9703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c: No bootstrap required, opened a new log
I20260929 16:09:52.866688 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:52.841280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:53485 (local address 127.0.0.1:42149)
0929 16:09:52.842085 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:52.842097 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:52.842140 (+ 43us) server_negotiation.cc:407] Connection header received
0929 16:09:52.843833 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:52.843856 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:52.844277 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:52.844605 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:52.846688 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:52.848125 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:52.850170 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:52.850979 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:52.851125 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:52.857043 (+ 5918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:52.857056 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:52.857078 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:52.857133 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:52.861477 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:52.862511 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:52.862521 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:52.862537 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:52.862751 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:52.863218 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:52.863223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:52.863226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:52.863982 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:52.864317 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:52.864752 (+ 435us) server_negotiation.cc:299] Negotiation successful
0929 16:09:52.865141 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":333,"threads_started":1}
I20260929 16:09:52.869503 9703 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18bf2564394d96a90d35a82ed2cf5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } }
I20260929 16:09:52.870039 9703 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:52.870167 9703 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea18bf2564394d96a90d35a82ed2cf5c, State: Initialized, Role: FOLLOWER
I20260929 16:09:52.870807 9703 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18bf2564394d96a90d35a82ed2cf5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } }
I20260929 16:09:52.871134 9703 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:52.871281 9703 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:52.871479 9703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:52.871721 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:09:52.873780 9703 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18bf2564394d96a90d35a82ed2cf5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } }
I20260929 16:09:52.874334 9703 leader_election.cc:302] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea18bf2564394d96a90d35a82ed2cf5c; no voters:
I20260929 16:09:52.875175 9703 leader_election.cc:288] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:52.875411 9710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:52.876328 9710 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [term 1 LEADER]: Becoming Leader. State: Replica: ea18bf2564394d96a90d35a82ed2cf5c, State: Running, Role: LEADER
I20260929 16:09:52.876955 9710 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18bf2564394d96a90d35a82ed2cf5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } }
I20260929 16:09:52.877687 9703 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:52.881942 9713 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea18bf2564394d96a90d35a82ed2cf5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18bf2564394d96a90d35a82ed2cf5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } } }
I20260929 16:09:52.882509 9713 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:52.887312 9712 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea18bf2564394d96a90d35a82ed2cf5c. Latest consensus state: current_term: 1 leader_uuid: "ea18bf2564394d96a90d35a82ed2cf5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18bf2564394d96a90d35a82ed2cf5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42149 } } }
I20260929 16:09:52.887730 9712 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:52.895077 9718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:52.901070 9718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:52.903072 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:52.878730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:60399 (local address 127.0.0.1:42149)
0929 16:09:52.878916 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:52.878921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:52.879497 (+ 576us) server_negotiation.cc:407] Connection header received
0929 16:09:52.879734 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:52.879741 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:52.879870 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:52.879980 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:52.882395 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:52.883852 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:52.886768 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:52.887678 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:52.887828 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:52.892672 (+ 4844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:52.892684 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:52.892692 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:52.892757 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:52.897335 (+ 4578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:52.898752 (+ 1417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:52.898760 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:52.898767 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:52.898988 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:52.899739 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:52.899747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:52.899753 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:52.900316 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:52.900683 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:52.902228 (+ 1545us) server_negotiation.cc:299] Negotiation successful
0929 16:09:52.902481 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 16:09:52.914275 9718 catalog_manager.cc:1347] Generated new cluster ID: 8ded28ebb62a4d978f1424fd941c984f
I20260929 16:09:52.914428 9718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:52.935329 9718 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:52.936971 9718 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:52.945535 9718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea18bf2564394d96a90d35a82ed2cf5c: Generated new TSK 0
I20260929 16:09:52.946561 9718 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:52.958631 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9636
I20260929 16:09:52.960057 9698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:53.060110 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 9636
W20260929 16:09:53.078349 9621 connection.cc:535] client connection to 127.0.0.1:42149 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42149): Connection reset by peer (error 104)
2026-09-29T16:09:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2210 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:53 dist-test-slave-x07d krb5kdc[9735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:53 dist-test-slave-x07d krb5kdc[9735](info): set up 2 sockets
Sep 29 16:09:53 dist-test-slave-x07d krb5kdc[9735](info): commencing operation
krb5kdc: starting...
W20260929 16:09:54.219013 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.005s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:54 dist-test-slave-x07d krb5kdc[9735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:09:54.505946 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:54.862115 9748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:54.862450 9748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:54.862672 9748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:54.871016 9748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:54.871141 9748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:54.871259 9748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:54.871372 9748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:54.871454 9748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:54.882560 9748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:54.883988 9748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:54.885375 9748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:54.898178 9756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:54.898205 9754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:54.898475 9753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:54.899122 9748 server_base.cc:1034] running on GCE node
I20260929 16:09:54.899987 9748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:54.901429 9748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:54.902637 9748 hybrid_clock.cc:648] HybridClock initialized: now 1790698194902595 us; error 33 us; skew 500 ppm
I20260929 16:09:54.902992 9748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:54 dist-test-slave-x07d krb5kdc[9735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698194, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:54.906401 9748 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:09:54.908107 9748 webserver.cc:460] Webserver started at http://127.0.0.1:36729/ using document root <none> and password file <none>
I20260929 16:09:54.908828 9748 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:54.908967 9748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:54.909288 9748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:54.912508 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "eaf22b8786c14472a50bf60b558641d9"
format_stamp: "Formatted at 2026-09-29 16:09:54 on dist-test-slave-x07d"
server_key: "2d222222d5710781b2f14a03a482d628"
server_key_iv: "f52685b0af7b4a3fe15c1b7776ac2ff0"
server_key_version: "encryptionkey@0"
I20260929 16:09:54.913367 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "eaf22b8786c14472a50bf60b558641d9"
format_stamp: "Formatted at 2026-09-29 16:09:54 on dist-test-slave-x07d"
server_key: "2d222222d5710781b2f14a03a482d628"
server_key_iv: "f52685b0af7b4a3fe15c1b7776ac2ff0"
server_key_version: "encryptionkey@0"
I20260929 16:09:54.919366 9748 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260929 16:09:54.923713 9763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:54.925343 9748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260929 16:09:54.925588 9748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "eaf22b8786c14472a50bf60b558641d9"
format_stamp: "Formatted at 2026-09-29 16:09:54 on dist-test-slave-x07d"
server_key: "2d222222d5710781b2f14a03a482d628"
server_key_iv: "f52685b0af7b4a3fe15c1b7776ac2ff0"
server_key_version: "encryptionkey@0"
I20260929 16:09:54.925834 9748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:54.947137 9748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:54.952540 9748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:54.952914 9748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:54.973416 9748 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36699
I20260929 16:09:54.973453 9814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36699 every 8 connection(s)
I20260929 16:09:54.974956 9748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:54.978886 9815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:54.984442 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 9748
I20260929 16:09:54.984642 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:54.985176 20966 external_mini_cluster.cc:1428] Setting key 07080808ff5b2dab98db60298ea8fc02
I20260929 16:09:54.991029 9815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9: Bootstrap starting.
I20260929 16:09:54.994241 9815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:54.995317 9815 log.cc:824] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:54.998965 9815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9: No bootstrap required, opened a new log
I20260929 16:09:54.999239 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:54.987361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:59407 (local address 127.0.0.1:36699)
0929 16:09:54.988190 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:54.988206 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:54.988251 (+ 45us) server_negotiation.cc:407] Connection header received
0929 16:09:54.990096 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:54.990121 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:54.990555 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:54.990937 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:54.991487 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:54.992794 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:54.994147 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:54.994896 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:54.995096 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:54.996462 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:54.996471 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:54.996485 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:54.997205 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:54.997301 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:54.997335 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:54.997508 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:54.997767 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 16:09:54.997952 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":341,"threads_started":1}
I20260929 16:09:55.003858 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:09:55.009766 9815 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf22b8786c14472a50bf60b558641d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } }
I20260929 16:09:55.010377 9815 raft_consensus.cc:374] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:55.010511 9815 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf22b8786c14472a50bf60b558641d9, State: Initialized, Role: FOLLOWER
I20260929 16:09:55.011188 9815 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf22b8786c14472a50bf60b558641d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } }
I20260929 16:09:55.011451 9815 raft_consensus.cc:388] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:55.011615 9815 raft_consensus.cc:482] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:55.011808 9815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:55.014154 9815 raft_consensus.cc:504] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf22b8786c14472a50bf60b558641d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } }
I20260929 16:09:55.014693 9815 leader_election.cc:302] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eaf22b8786c14472a50bf60b558641d9; no voters:
I20260929 16:09:55.015379 9815 leader_election.cc:288] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:55.015551 9825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:55.016323 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:55.009246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:48351 (local address 127.0.0.1:36699)
0929 16:09:55.009485 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:55.009492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:55.009812 (+ 320us) server_negotiation.cc:407] Connection header received
0929 16:09:55.010076 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:55.010080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:55.010148 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:55.010242 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:55.010883 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:55.011885 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:55.013146 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:55.013630 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:55.013769 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:55.015067 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:55.015077 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:55.015084 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:55.015111 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:55.015208 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:55.015236 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:55.015392 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:55.015801 (+ 409us) server_negotiation.cc:299] Negotiation successful
0929 16:09:55.015880 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 16:09:55.017807 9815 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:55.022360 9825 raft_consensus.cc:686] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [term 1 LEADER]: Becoming Leader. State: Replica: eaf22b8786c14472a50bf60b558641d9, State: Running, Role: LEADER
I20260929 16:09:55.023092 9825 consensus_queue.cc:237] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf22b8786c14472a50bf60b558641d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } }
I20260929 16:09:55.027244 9826 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eaf22b8786c14472a50bf60b558641d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf22b8786c14472a50bf60b558641d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } } }
I20260929 16:09:55.027721 9826 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:55.031032 9827 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eaf22b8786c14472a50bf60b558641d9. Latest consensus state: current_term: 1 leader_uuid: "eaf22b8786c14472a50bf60b558641d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eaf22b8786c14472a50bf60b558641d9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36699 } } }
I20260929 16:09:55.031483 9827 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:55.032307 9841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:55.040303 9841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:55.051545 9841 catalog_manager.cc:1347] Generated new cluster ID: cce1701b29934d98b3180c097cacac17
I20260929 16:09:55.051725 9841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:55.067195 9841 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:55.069036 9841 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:55.074992 9841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eaf22b8786c14472a50bf60b558641d9: Generated new TSK 0
I20260929 16:09:55.076040 9841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:55.082755 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9748
W20260929 16:09:55.083842 9810 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.53:59407
W20260929 16:09:55.084715 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9748: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:09:55.084889 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 9748
W20260929 16:09:55.102303 9733 connection.cc:535] client connection to 127.0.0.1:36699 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36699): Connection reset by peer (error 104)
2026-09-29T16:09:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2023 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:55 dist-test-slave-x07d krb5kdc[9846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:55 dist-test-slave-x07d krb5kdc[9846](info): set up 2 sockets
Sep 29 16:09:55 dist-test-slave-x07d krb5kdc[9846](info): commencing operation
krb5kdc: starting...
W20260929 16:09:56.218595 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.011s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:56 dist-test-slave-x07d krb5kdc[9846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:09:56.495484 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44045
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:56.854158 9859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:56.854889 9859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:56.855105 9859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:56.864133 9859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:56.864282 9859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:56.864383 9859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:56.864511 9859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:56.864586 9859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:56.876252 9859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:56.877838 9859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:56.879398 9859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:56.893328 9865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:56.893375 9867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:56.893391 9864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:56.895073 9859 server_base.cc:1034] running on GCE node
I20260929 16:09:56.896036 9859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:56.897737 9859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:56.898965 9859 hybrid_clock.cc:648] HybridClock initialized: now 1790698196898903 us; error 58 us; skew 500 ppm
I20260929 16:09:56.899376 9859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:56 dist-test-slave-x07d krb5kdc[9846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:56.903273 9859 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:09:56.905030 9859 webserver.cc:460] Webserver started at http://127.0.0.1:43949/ using document root <none> and password file <none>
I20260929 16:09:56.905800 9859 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:56.905951 9859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:56.906395 9859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:56.909727 9859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "2326fd3a67c94dfdab9d47c9684e2505"
format_stamp: "Formatted at 2026-09-29 16:09:56 on dist-test-slave-x07d"
server_key: "a0c80b6fa369d5fe844e90d7fe0f5646"
server_key_iv: "177e20f522577d20a54045b13af02413"
server_key_version: "encryptionkey@0"
I20260929 16:09:56.910715 9859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "2326fd3a67c94dfdab9d47c9684e2505"
format_stamp: "Formatted at 2026-09-29 16:09:56 on dist-test-slave-x07d"
server_key: "a0c80b6fa369d5fe844e90d7fe0f5646"
server_key_iv: "177e20f522577d20a54045b13af02413"
server_key_version: "encryptionkey@0"
I20260929 16:09:56.916712 9859 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 16:09:56.920971 9874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:56.922734 9859 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260929 16:09:56.922977 9859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "2326fd3a67c94dfdab9d47c9684e2505"
format_stamp: "Formatted at 2026-09-29 16:09:56 on dist-test-slave-x07d"
server_key: "a0c80b6fa369d5fe844e90d7fe0f5646"
server_key_iv: "177e20f522577d20a54045b13af02413"
server_key_version: "encryptionkey@0"
I20260929 16:09:56.923223 9859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:56.929858 9859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:56.935123 9859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:56.935488 9859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:56.957412 9859 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44045
I20260929 16:09:56.957475 9925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44045 every 8 connection(s)
I20260929 16:09:56.959029 9859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:56.962320 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 9859
I20260929 16:09:56.962527 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:56.963011 20966 external_mini_cluster.cc:1428] Setting key 8ae221458943ffd4ae64bafdd4257c6c
I20260929 16:09:56.963413 9926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:56.972644 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:56.965053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:46769 (local address 127.0.0.1:44045)
0929 16:09:56.965915 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:56.965931 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:56.965989 (+ 58us) server_negotiation.cc:407] Connection header received
0929 16:09:56.968398 (+ 2409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:56.968431 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:56.968915 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:56.969412 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:56.970546 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:56.970554 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:56.970571 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:56.970590 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:56.970667 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:56.970709 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:56.970943 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:56.971102 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 16:09:56.971329 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":364,"threads_started":1}
I20260929 16:09:56.976886 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:09:56.978819 9926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505: Bootstrap starting.
I20260929 16:09:56.983366 9926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:56.984912 9926 log.cc:824] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:56.985651 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:56.982501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:60081 (local address 127.0.0.1:44045)
0929 16:09:56.982748 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:56.982758 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:56.983310 (+ 552us) server_negotiation.cc:407] Connection header received
0929 16:09:56.983564 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:56.983568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:56.983630 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:56.983724 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:56.984816 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:56.984821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:56.984824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:09:56.984841 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:56.984893 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:09:56.984917 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:56.985074 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:56.985206 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 16:09:56.985264 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 16:09:56.988190 9926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505: No bootstrap required, opened a new log
I20260929 16:09:56.996342 9926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2326fd3a67c94dfdab9d47c9684e2505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } }
I20260929 16:09:56.996793 9926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:56.996906 9926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2326fd3a67c94dfdab9d47c9684e2505, State: Initialized, Role: FOLLOWER
I20260929 16:09:56.997537 9926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2326fd3a67c94dfdab9d47c9684e2505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } }
I20260929 16:09:56.997784 9926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:56.997947 9926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:56.998220 9926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:57.000584 9926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2326fd3a67c94dfdab9d47c9684e2505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } }
I20260929 16:09:57.001271 9926 leader_election.cc:302] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2326fd3a67c94dfdab9d47c9684e2505; no voters:
I20260929 16:09:57.002069 9926 leader_election.cc:288] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:57.002262 9936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:57.002974 9936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [term 1 LEADER]: Becoming Leader. State: Replica: 2326fd3a67c94dfdab9d47c9684e2505, State: Running, Role: LEADER
I20260929 16:09:57.003608 9936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2326fd3a67c94dfdab9d47c9684e2505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } }
I20260929 16:09:57.004421 9926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:57.006490 9937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2326fd3a67c94dfdab9d47c9684e2505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2326fd3a67c94dfdab9d47c9684e2505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } } }
I20260929 16:09:57.006886 9937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:57.007779 9943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:57.012071 9938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2326fd3a67c94dfdab9d47c9684e2505. Latest consensus state: current_term: 1 leader_uuid: "2326fd3a67c94dfdab9d47c9684e2505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2326fd3a67c94dfdab9d47c9684e2505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } } }
I20260929 16:09:57.012480 9938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:57.016429 9943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:57.028545 9943 catalog_manager.cc:1347] Generated new cluster ID: 8a1071d056a84e3aa6873d46b9ffcac4
I20260929 16:09:57.028710 9943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:57.052395 9943 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:57.053031 9943 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:57.067343 9943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2326fd3a67c94dfdab9d47c9684e2505: Generated new TSK 0
I20260929 16:09:57.068594 9943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:57.083639 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9859
W20260929 16:09:57.085166 9921 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.53:46769
W20260929 16:09:57.085978 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9859: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:09:57.086175 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 9859
2026-09-29T16:09:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2001 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:57 dist-test-slave-x07d krb5kdc[9957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:57 dist-test-slave-x07d krb5kdc[9957](info): set up 2 sockets
Sep 29 16:09:57 dist-test-slave-x07d krb5kdc[9957](info): commencing operation
krb5kdc: starting...
W20260929 16:09:58.258610 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.001s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:09:58 dist-test-slave-x07d krb5kdc[9957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:09:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:09:58.538394 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32885
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:09:58.882966 9970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:09:58.883289 9970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:09:58.883494 9970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:09:58.891026 9970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:09:58.891161 9970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:09:58.891280 9970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:09:58.891398 9970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:09:58.891455 9970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:09:58.904554 9970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32885
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:09:58.905992 9970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:09:58.907476 9970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:09:58.919793 9976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:58.919728 9978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:09:58.919824 9975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:09:58.920702 9970 server_base.cc:1034] running on GCE node
I20260929 16:09:58.921530 9970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:09:58.923226 9970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:09:58.924468 9970 hybrid_clock.cc:648] HybridClock initialized: now 1790698198924417 us; error 50 us; skew 500 ppm
I20260929 16:09:58.924835 9970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:09:58 dist-test-slave-x07d krb5kdc[9957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:09:58.928253 9970 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:09:58.929870 9970 webserver.cc:460] Webserver started at http://127.0.0.1:38827/ using document root <none> and password file <none>
I20260929 16:09:58.930603 9970 fs_manager.cc:359] Metadata directory not provided
I20260929 16:09:58.930708 9970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:09:58.930979 9970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:09:58.933996 9970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "a0d539272f7c43e49a6400111312f7c5"
format_stamp: "Formatted at 2026-09-29 16:09:58 on dist-test-slave-x07d"
server_key: "f82c46e953a1bc3b21c44f7ab2ae948e"
server_key_iv: "2696b8a9d495abd55f6628678517440f"
server_key_version: "encryptionkey@0"
I20260929 16:09:58.934834 9970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "a0d539272f7c43e49a6400111312f7c5"
format_stamp: "Formatted at 2026-09-29 16:09:58 on dist-test-slave-x07d"
server_key: "f82c46e953a1bc3b21c44f7ab2ae948e"
server_key_iv: "2696b8a9d495abd55f6628678517440f"
server_key_version: "encryptionkey@0"
I20260929 16:09:58.940301 9970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:09:58.944351 9985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:09:58.946147 9970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 16:09:58.946357 9970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "a0d539272f7c43e49a6400111312f7c5"
format_stamp: "Formatted at 2026-09-29 16:09:58 on dist-test-slave-x07d"
server_key: "f82c46e953a1bc3b21c44f7ab2ae948e"
server_key_iv: "2696b8a9d495abd55f6628678517440f"
server_key_version: "encryptionkey@0"
I20260929 16:09:58.946558 9970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:09:58.967136 9970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:09:58.971658 9970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:09:58.971983 9970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:09:58.992276 9970 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32885
I20260929 16:09:58.992313 10036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32885 every 8 connection(s)
I20260929 16:09:58.993745 9970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:09:58.997040 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 9970
I20260929 16:09:58.997231 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:09:58.997786 20966 external_mini_cluster.cc:1428] Setting key d2066cc3798b96110bee65509884bea4
I20260929 16:09:58.997920 10037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:09:59.011761 10037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5: Bootstrap starting.
Sep 29 16:09:59 dist-test-slave-x07d krb5kdc[9957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 16:09:59.015868 10037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5: Neither blocks nor log segments found. Creating new log.
I20260929 16:09:59.017138 10037 log.cc:824] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5: Log is configured to *not* fsync() on all Append() calls
I20260929 16:09:59.020432 10037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5: No bootstrap required, opened a new log
I20260929 16:09:59.022063 10040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:58.999687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:57401 (local address 127.0.0.1:32885)
0929 16:09:59.000465 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:59.000477 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:59.000522 (+ 45us) server_negotiation.cc:407] Connection header received
0929 16:09:59.002936 (+ 2414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:59.002961 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:59.003418 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:59.003857 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:59.005409 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:59.006806 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:59.008108 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:59.008854 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:59.009081 (+ 227us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:59.013913 (+ 4832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:59.013922 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:59.013936 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:59.013980 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:59.017514 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:59.018539 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:59.018546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:59.018554 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:59.018736 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:59.019236 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:59.019246 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:59.019248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:59.019794 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:59.020143 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:59.020642 (+ 499us) server_negotiation.cc:299] Negotiation successful
0929 16:09:59.020972 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":319,"threads_started":1}
I20260929 16:09:59.026170 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:09:59.030284 10037 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d539272f7c43e49a6400111312f7c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32885 } }
I20260929 16:09:59.030931 10037 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:09:59.031066 10037 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0d539272f7c43e49a6400111312f7c5, State: Initialized, Role: FOLLOWER
I20260929 16:09:59.032161 10037 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d539272f7c43e49a6400111312f7c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32885 } }
I20260929 16:09:59.032600 10037 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:09:59.032783 10037 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:09:59.032994 10037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:09:59.036410 10037 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d539272f7c43e49a6400111312f7c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32885 } }
I20260929 16:09:59.037088 10037 leader_election.cc:302] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0d539272f7c43e49a6400111312f7c5; no voters:
I20260929 16:09:59.037747 10037 leader_election.cc:288] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:09:59.037910 10047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:09:59.040207 10037 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:09:59.040777 10047 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [term 1 LEADER]: Becoming Leader. State: Replica: a0d539272f7c43e49a6400111312f7c5, State: Running, Role: LEADER
I20260929 16:09:59.041440 10047 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d539272f7c43e49a6400111312f7c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32885 } }
I20260929 16:09:59.045456 10049 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0d539272f7c43e49a6400111312f7c5. Latest consensus state: current_term: 1 leader_uuid: "a0d539272f7c43e49a6400111312f7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d539272f7c43e49a6400111312f7c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32885 } } }
I20260929 16:09:59.045900 10049 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:59.046514 10048 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0d539272f7c43e49a6400111312f7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d539272f7c43e49a6400111312f7c5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32885 } } }
I20260929 16:09:59.046823 10048 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5 [sys.catalog]: This master's current role is: LEADER
I20260929 16:09:59.052651 10055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:09:59.054279 10040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:09:59.030734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:40581 (local address 127.0.0.1:32885)
0929 16:09:59.030972 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0929 16:09:59.030977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:09:59.033168 (+ 2191us) server_negotiation.cc:407] Connection header received
0929 16:09:59.033323 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:09:59.033336 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:09:59.033450 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:09:59.033579 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:09:59.035204 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:59.036449 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:59.038210 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:09:59.039129 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:09:59.039274 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:09:59.043221 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:09:59.043233 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:09:59.043242 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:09:59.043322 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:09:59.049486 (+ 6164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:59.051194 (+ 1708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:59.051203 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:59.051210 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:59.051429 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:09:59.052222 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:09:59.052231 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:09:59.052236 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:09:59.052732 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:09:59.053105 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0929 16:09:59.053565 (+ 460us) server_negotiation.cc:299] Negotiation successful
0929 16:09:59.053787 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":62}
I20260929 16:09:59.060482 10055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:09:59.071307 10055 catalog_manager.cc:1347] Generated new cluster ID: 77653f879df04da788e7f83cac7b6d1b
I20260929 16:09:59.071491 10055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:09:59.086642 10055 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:09:59.088091 10055 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:09:59.103580 10055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0d539272f7c43e49a6400111312f7c5: Generated new TSK 0
I20260929 16:09:59.104525 10055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:09:59.117424 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 9970
I20260929 16:09:59.118811 10032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:09:59.215539 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 9970
W20260929 16:09:59.233309 9955 connection.cc:535] client connection to 127.0.0.1:32885 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32885): Connection reset by peer (error 104)
2026-09-29T16:09:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2134 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:09:59 dist-test-slave-x07d krb5kdc[10069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:09:59 dist-test-slave-x07d krb5kdc[10069](info): set up 2 sockets
Sep 29 16:09:59 dist-test-slave-x07d krb5kdc[10069](info): commencing operation
krb5kdc: starting...
W20260929 16:10:00.338653 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:10:00 dist-test-slave-x07d krb5kdc[10069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:10:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:10:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:10:00.618129 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45645
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:10:00.977161 10082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:00.977592 10082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:00.977777 10082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:00.985656 10082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:00.985872 10082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:00.985996 10082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:00.986131 10082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:00.986227 10082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:00.997008 10082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45645
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:00.998538 10082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:01.000079 10082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:01.013993 10087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:01.013949 10090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:01.014052 10088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:01.015043 10082 server_base.cc:1034] running on GCE node
I20260929 16:10:01.016067 10082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:01.017792 10082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:01.019076 10082 hybrid_clock.cc:648] HybridClock initialized: now 1790698201019040 us; error 81 us; skew 500 ppm
I20260929 16:10:01.019518 10082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:01 dist-test-slave-x07d krb5kdc[10069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:01.023206 10082 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:10:01.025014 10082 webserver.cc:460] Webserver started at http://127.0.0.1:44245/ using document root <none> and password file <none>
I20260929 16:10:01.025745 10082 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:01.025904 10082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:01.026288 10082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:01.029498 10082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "e4699386199244b3aef737acec716355"
format_stamp: "Formatted at 2026-09-29 16:10:01 on dist-test-slave-x07d"
server_key: "7decfe75c42ac11a0a72b09ec28bb8ed"
server_key_iv: "4dff75d24a882892a1e10b9c3d1a6b43"
server_key_version: "encryptionkey@0"
I20260929 16:10:01.030473 10082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "e4699386199244b3aef737acec716355"
format_stamp: "Formatted at 2026-09-29 16:10:01 on dist-test-slave-x07d"
server_key: "7decfe75c42ac11a0a72b09ec28bb8ed"
server_key_iv: "4dff75d24a882892a1e10b9c3d1a6b43"
server_key_version: "encryptionkey@0"
I20260929 16:10:01.036475 10082 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 16:10:01.040643 10097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:01.042372 10082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:10:01.042593 10082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "e4699386199244b3aef737acec716355"
format_stamp: "Formatted at 2026-09-29 16:10:01 on dist-test-slave-x07d"
server_key: "7decfe75c42ac11a0a72b09ec28bb8ed"
server_key_iv: "4dff75d24a882892a1e10b9c3d1a6b43"
server_key_version: "encryptionkey@0"
I20260929 16:10:01.042826 10082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:01.060405 10082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:01.065260 10082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:01.065583 10082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:01.085434 10082 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45645
I20260929 16:10:01.085474 10148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45645 every 8 connection(s)
I20260929 16:10:01.086932 10082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:10:01.090987 10149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:01.095335 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 10082
I20260929 16:10:01.095533 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:10:01.096078 20966 external_mini_cluster.cc:1428] Setting key 57c6d45fee00eb3020589ab4e8a192c7
I20260929 16:10:01.105089 10149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355: Bootstrap starting.
I20260929 16:10:01.108971 10149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:01.110177 10149 log.cc:824] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:01.110643 10152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:01.098283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:55493 (local address 127.0.0.1:45645)
0929 16:10:01.099120 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:01.099132 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:01.099189 (+ 57us) server_negotiation.cc:407] Connection header received
0929 16:10:01.100854 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:01.100880 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:01.101345 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:01.101706 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:01.102404 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:01.103878 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:01.105834 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:01.106628 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:01.106780 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:01.108148 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:01.108155 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:01.108168 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:01.108710 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:01.108780 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:01.108804 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:01.108943 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:01.109316 (+ 373us) server_negotiation.cc:299] Negotiation successful
0929 16:10:01.109523 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":383,"threads_started":1}
I20260929 16:10:01.113097 10149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355: No bootstrap required, opened a new log
I20260929 16:10:01.114378 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:10:01.123926 10149 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4699386199244b3aef737acec716355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } }
I20260929 16:10:01.124491 10149 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:10:01.124635 10149 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4699386199244b3aef737acec716355, State: Initialized, Role: FOLLOWER
I20260929 16:10:01.125447 10149 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4699386199244b3aef737acec716355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } }
I20260929 16:10:01.125794 10149 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:10:01.125953 10149 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:10:01.126170 10149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:01.128144 10152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:01.119818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:49985 (local address 127.0.0.1:45645)
0929 16:10:01.120031 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:01.120037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:01.120611 (+ 574us) server_negotiation.cc:407] Connection header received
0929 16:10:01.120875 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:01.120880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:01.120964 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:01.121061 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:01.121692 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:01.122672 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:01.124574 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:01.125331 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:01.125491 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:01.126966 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:01.126972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:01.126975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:01.126998 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:01.127072 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:01.127094 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:01.127199 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:01.127535 (+ 336us) server_negotiation.cc:299] Negotiation successful
0929 16:10:01.127608 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:10:01.129287 10149 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4699386199244b3aef737acec716355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } }
I20260929 16:10:01.129953 10149 leader_election.cc:302] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4699386199244b3aef737acec716355; no voters:
I20260929 16:10:01.130654 10149 leader_election.cc:288] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:10:01.130823 10159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:10:01.131559 10159 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [term 1 LEADER]: Becoming Leader. State: Replica: e4699386199244b3aef737acec716355, State: Running, Role: LEADER
I20260929 16:10:01.132184 10159 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4699386199244b3aef737acec716355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } }
I20260929 16:10:01.133133 10149 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:01.135223 10160 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4699386199244b3aef737acec716355" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4699386199244b3aef737acec716355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } } }
I20260929 16:10:01.135705 10160 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:01.135584 10161 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4699386199244b3aef737acec716355. Latest consensus state: current_term: 1 leader_uuid: "e4699386199244b3aef737acec716355" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4699386199244b3aef737acec716355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45645 } } }
I20260929 16:10:01.135876 10161 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:01.137451 10167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:10:01.143213 10167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:10:01.154117 10167 catalog_manager.cc:1347] Generated new cluster ID: 57a0ff90ed844c9a9fbcc9aea44441ab
I20260929 16:10:01.154312 10167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:10:01.179401 10167 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:10:01.181388 10167 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:10:01.191615 10167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4699386199244b3aef737acec716355: Generated new TSK 0
I20260929 16:10:01.192698 10167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:10:01.209038 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10082
W20260929 16:10:01.210690 10144 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.53:55493
W20260929 16:10:01.211560 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10082: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:10:01.211719 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 10082
W20260929 16:10:01.229730 10067 connection.cc:535] client connection to 127.0.0.1:45645 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45645): Connection reset by peer (error 104)
2026-09-29T16:10:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1991 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:10:01 dist-test-slave-x07d krb5kdc[10180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:10:01 dist-test-slave-x07d krb5kdc[10180](info): set up 2 sockets
Sep 29 16:10:01 dist-test-slave-x07d krb5kdc[10180](info): commencing operation
krb5kdc: starting...
W20260929 16:10:02.358875 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:10:02 dist-test-slave-x07d krb5kdc[10180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:10:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:10:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 16:10:02.645987 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:10:03.007170 10193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:03.007513 10193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:03.007613 10193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:03.015308 10193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:03.015451 10193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:03.015501 10193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:03.015556 10193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:03.015604 10193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:03.028299 10193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:03.029649 10193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:03.031005 10193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:03.044456 10198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:03.044432 10199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:03.044394 10201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:03.045042 10193 server_base.cc:1034] running on GCE node
I20260929 16:10:03.046105 10193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:03.047744 10193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:03.049028 10193 hybrid_clock.cc:648] HybridClock initialized: now 1790698203048982 us; error 81 us; skew 500 ppm
I20260929 16:10:03.049470 10193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:03 dist-test-slave-x07d krb5kdc[10180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698203, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:03.053102 10193 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 16:10:03.055080 10193 webserver.cc:460] Webserver started at http://127.0.0.1:34569/ using document root <none> and password file <none>
I20260929 16:10:03.055847 10193 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:03.055959 10193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:03.056218 10193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:03.059701 10193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "814935aa080444009a1536246d195159"
format_stamp: "Formatted at 2026-09-29 16:10:03 on dist-test-slave-x07d"
server_key: "0d6476895f5a894de24226502edf7175"
server_key_iv: "f765ebb0223efba280e1d72e7df4abdc"
server_key_version: "encryptionkey@0"
I20260929 16:10:03.060544 10193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "814935aa080444009a1536246d195159"
format_stamp: "Formatted at 2026-09-29 16:10:03 on dist-test-slave-x07d"
server_key: "0d6476895f5a894de24226502edf7175"
server_key_iv: "f765ebb0223efba280e1d72e7df4abdc"
server_key_version: "encryptionkey@0"
I20260929 16:10:03.066443 10193 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 16:10:03.070286 10208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:03.071771 10193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:10:03.071986 10193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "814935aa080444009a1536246d195159"
format_stamp: "Formatted at 2026-09-29 16:10:03 on dist-test-slave-x07d"
server_key: "0d6476895f5a894de24226502edf7175"
server_key_iv: "f765ebb0223efba280e1d72e7df4abdc"
server_key_version: "encryptionkey@0"
I20260929 16:10:03.072170 10193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:03.078464 10193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:03.087452 10193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:03.087754 10193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:03.109403 10193 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37225
I20260929 16:10:03.109457 10259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37225 every 8 connection(s)
I20260929 16:10:03.111152 10193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:10:03.115456 10260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:03.116138 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 10193
I20260929 16:10:03.116322 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:10:03.116823 20966 external_mini_cluster.cc:1428] Setting key 274e5ca37570a367c8680c7a04f55b5f
I20260929 16:10:03.127192 10263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:03.119004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:52599 (local address 127.0.0.1:37225)
0929 16:10:03.119909 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:03.119926 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:03.119983 (+ 57us) server_negotiation.cc:407] Connection header received
0929 16:10:03.122764 (+ 2781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:03.122803 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:03.123371 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:03.123880 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:03.125051 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:03.125060 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:03.125078 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:03.125097 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:03.125174 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:03.125210 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:03.125455 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:03.125585 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 16:10:03.125810 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":344,"threads_started":1}
I20260929 16:10:03.131155 10260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159: Bootstrap starting.
I20260929 16:10:03.132145 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:10:03.135439 10260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:03.137121 10260 log.cc:824] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:03.141069 10263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:03.138367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.53:58221 (local address 127.0.0.1:37225)
0929 16:10:03.138580 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:03.138586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:03.138974 (+ 388us) server_negotiation.cc:407] Connection header received
0929 16:10:03.139133 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:03.139139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:03.139202 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:03.139324 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:03.140344 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:03.140350 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:03.140355 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:03.140374 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:03.140421 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:03.140441 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:03.140565 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:03.140722 (+ 157us) server_negotiation.cc:299] Negotiation successful
0929 16:10:03.140790 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 16:10:03.141280 10260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159: No bootstrap required, opened a new log
I20260929 16:10:03.149556 10260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814935aa080444009a1536246d195159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260929 16:10:03.149996 10260 raft_consensus.cc:374] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:10:03.150174 10260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 814935aa080444009a1536246d195159, State: Initialized, Role: FOLLOWER
I20260929 16:10:03.150810 10260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814935aa080444009a1536246d195159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260929 16:10:03.151091 10260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:10:03.151266 10260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:10:03.151450 10260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:03.153874 10260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814935aa080444009a1536246d195159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260929 16:10:03.154484 10260 leader_election.cc:302] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 814935aa080444009a1536246d195159; no voters:
I20260929 16:10:03.155124 10260 leader_election.cc:288] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:10:03.155339 10270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:10:03.156180 10270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [term 1 LEADER]: Becoming Leader. State: Replica: 814935aa080444009a1536246d195159, State: Running, Role: LEADER
I20260929 16:10:03.156932 10270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814935aa080444009a1536246d195159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260929 16:10:03.157697 10260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:03.160382 10271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "814935aa080444009a1536246d195159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814935aa080444009a1536246d195159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } } }
I20260929 16:10:03.160843 10271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:03.161607 10272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 814935aa080444009a1536246d195159. Latest consensus state: current_term: 1 leader_uuid: "814935aa080444009a1536246d195159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814935aa080444009a1536246d195159" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } } }
I20260929 16:10:03.162267 10272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:03.171288 10279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:10:03.176137 10279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:10:03.185341 10279 catalog_manager.cc:1347] Generated new cluster ID: 65c2758df22b4846af25a69f0016062c
I20260929 16:10:03.185599 10279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:10:03.208070 10279 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:10:03.208666 10279 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:10:03.229245 10279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 814935aa080444009a1536246d195159: Generated new TSK 0
I20260929 16:10:03.230480 10279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:10:03.240247 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10193
W20260929 16:10:03.241802 10255 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.53:52599
W20260929 16:10:03.242733 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10193: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:10:03.242903 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 10193
2026-09-29T16:10:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2030 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:10:03 dist-test-slave-x07d krb5kdc[10291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:10:03 dist-test-slave-x07d krb5kdc[10291](info): set up 2 sockets
Sep 29 16:10:03 dist-test-slave-x07d krb5kdc[10291](info): commencing operation
krb5kdc: starting...
W20260929 16:10:04.414633 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:10:04 dist-test-slave-x07d krb5kdc[10291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:10:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:10:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:10:04.708133 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:35411
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:35905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:35411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:10:05.055163 10304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:05.055469 10304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:05.055637 10304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:05.063472 10304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:05.063599 10304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:05.063727 10304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:05.063861 10304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:05.063946 10304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:05.074232 10304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:35905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:35411
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:35411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:05.075661 10304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:05.077121 10304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:05.090862 10309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:05.090845 10310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:05.090904 10312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:05.091722 10304 server_base.cc:1034] running on GCE node
I20260929 16:10:05.092732 10304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:05.094684 10304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:05.095975 10304 hybrid_clock.cc:648] HybridClock initialized: now 1790698205095907 us; error 61 us; skew 500 ppm
I20260929 16:10:05.096391 10304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:05 dist-test-slave-x07d krb5kdc[10291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698205, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:05.100183 10304 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:10:05.101991 10304 webserver.cc:460] Webserver started at http://127.20.121.190:41473/ using document root <none> and password file <none>
I20260929 16:10:05.102931 10304 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:05.103098 10304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:05.103425 10304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:05.106896 10304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "f4987434a17445f192e5ef3449d9b5ca"
format_stamp: "Formatted at 2026-09-29 16:10:05 on dist-test-slave-x07d"
server_key: "adb413de3c7a049f923da54f412d18ff"
server_key_iv: "6e5b3bdefd50e875298db22a28731612"
server_key_version: "encryptionkey@0"
I20260929 16:10:05.107803 10304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "f4987434a17445f192e5ef3449d9b5ca"
format_stamp: "Formatted at 2026-09-29 16:10:05 on dist-test-slave-x07d"
server_key: "adb413de3c7a049f923da54f412d18ff"
server_key_iv: "6e5b3bdefd50e875298db22a28731612"
server_key_version: "encryptionkey@0"
I20260929 16:10:05.113790 10304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 16:10:05.117755 10319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:05.119338 10304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 16:10:05.119597 10304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "f4987434a17445f192e5ef3449d9b5ca"
format_stamp: "Formatted at 2026-09-29 16:10:05 on dist-test-slave-x07d"
server_key: "adb413de3c7a049f923da54f412d18ff"
server_key_iv: "6e5b3bdefd50e875298db22a28731612"
server_key_version: "encryptionkey@0"
I20260929 16:10:05.119825 10304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:05.151353 10304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:05.156175 10304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:05.156508 10304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:05.176828 10304 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:35411
I20260929 16:10:05.176905 10370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:35411 every 8 connection(s)
I20260929 16:10:05.178409 10304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:10:05.182621 10371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:05.185539 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 10304
I20260929 16:10:05.185730 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:10:05.186324 20966 external_mini_cluster.cc:1428] Setting key 879e39f416502eb5b8178f656b0732d5
I20260929 16:10:05.195676 10371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca: Bootstrap starting.
I20260929 16:10:05.199970 10371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:05.201645 10371 log.cc:824] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca: Log is configured to *not* fsync() on all Append() calls
Sep 29 16:10:05 dist-test-slave-x07d krb5kdc[10291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:10:05.205839 10371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca: No bootstrap required, opened a new log
I20260929 16:10:05.214272 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:05.188529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51567 (local address 127.20.121.190:35411)
0929 16:10:05.189414 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:05.189426 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:05.189479 (+ 53us) server_negotiation.cc:407] Connection header received
0929 16:10:05.191325 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:05.191353 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:05.191804 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:05.192268 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:05.193946 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:05.195811 (+ 1865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:05.197455 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:05.198423 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:05.198683 (+ 260us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:05.204681 (+ 5998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:05.204695 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:05.204717 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:05.204782 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:05.208861 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:05.210216 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:05.210252 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:05.210259 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:05.210477 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:05.211051 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:05.211055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:05.211057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:05.211731 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:05.212000 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:05.212552 (+ 552us) server_negotiation.cc:299] Negotiation successful
0929 16:10:05.212954 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":391,"threads_started":1}
I20260929 16:10:05.215297 10371 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4987434a17445f192e5ef3449d9b5ca" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35411 } }
I20260929 16:10:05.215824 10371 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:10:05.215955 10371 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4987434a17445f192e5ef3449d9b5ca, State: Initialized, Role: FOLLOWER
I20260929 16:10:05.216642 10371 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4987434a17445f192e5ef3449d9b5ca" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35411 } }
I20260929 16:10:05.216969 10371 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:10:05.217155 10371 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:10:05.217367 10371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:05.219148 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:10:05.220705 10371 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4987434a17445f192e5ef3449d9b5ca" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35411 } }
I20260929 16:10:05.221366 10371 leader_election.cc:302] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4987434a17445f192e5ef3449d9b5ca; no voters:
I20260929 16:10:05.222150 10371 leader_election.cc:288] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:10:05.222285 10376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:10:05.222988 10376 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [term 1 LEADER]: Becoming Leader. State: Replica: f4987434a17445f192e5ef3449d9b5ca, State: Running, Role: LEADER
I20260929 16:10:05.223707 10376 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4987434a17445f192e5ef3449d9b5ca" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35411 } }
I20260929 16:10:05.225780 10376 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4987434a17445f192e5ef3449d9b5ca. Latest consensus state: current_term: 1 leader_uuid: "f4987434a17445f192e5ef3449d9b5ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4987434a17445f192e5ef3449d9b5ca" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35411 } } }
I20260929 16:10:05.226186 10376 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:05.226652 10379 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4987434a17445f192e5ef3449d9b5ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4987434a17445f192e5ef3449d9b5ca" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 35411 } } }
I20260929 16:10:05.227059 10379 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:05.236223 10371 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:05.236842 10382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:10:05.242548 10382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:10:05.260912 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:05.236277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56005 (local address 127.20.121.190:35411)
0929 16:10:05.236536 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:05.236546 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:05.237010 (+ 464us) server_negotiation.cc:407] Connection header received
0929 16:10:05.237311 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:05.237317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:05.237401 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:05.237515 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:05.240237 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:05.241924 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:05.244638 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:05.245700 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:05.245874 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:05.250514 (+ 4640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:05.250526 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:05.250534 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:05.250600 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:05.255721 (+ 5121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:05.257216 (+ 1495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:05.257226 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:05.257233 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:05.257445 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:05.258253 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:05.258263 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:05.258269 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:05.258843 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:05.259185 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:05.260151 (+ 966us) server_negotiation.cc:299] Negotiation successful
0929 16:10:05.260397 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 16:10:05.270224 10382 catalog_manager.cc:1347] Generated new cluster ID: e3af7384427a4141a1ca2e1a29e80673
I20260929 16:10:05.270394 10382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:10:05.296293 10382 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:10:05.298353 10382 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:10:05.314671 10382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4987434a17445f192e5ef3449d9b5ca: Generated new TSK 0
I20260929 16:10:05.315905 10382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:10:05.323359 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10304
I20260929 16:10:05.324656 10366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:10:05.421576 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 10304
W20260929 16:10:05.439976 10289 connection.cc:535] client connection to 127.20.121.190:35411 recv error: Network error: failed to read from TLS socket (remote: 127.20.121.190:35411): Connection reset by peer (error 104)
2026-09-29T16:10:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2178 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:10:05 dist-test-slave-x07d krb5kdc[10403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:10:05 dist-test-slave-x07d krb5kdc[10403](info): set up 2 sockets
Sep 29 16:10:05 dist-test-slave-x07d krb5kdc[10403](info): commencing operation
krb5kdc: starting...
W20260929 16:10:06.614585 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:10:06 dist-test-slave-x07d krb5kdc[10403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:10:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:10:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:10:06.894443 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:45243
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:45179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:45243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:10:07.256620 10416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:07.257031 10416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:07.257196 10416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:07.265017 10416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:07.265152 10416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:07.265230 10416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:07.265331 10416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:07.265437 10416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:07.276119 10416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:45179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:45243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:45243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:07.277539 10416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:07.278980 10416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:07.291679 10421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:07.291746 10424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:07.291642 10422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:07.292821 10416 server_base.cc:1034] running on GCE node
I20260929 16:10:07.293709 10416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:07.295341 10416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:07.296600 10416 hybrid_clock.cc:648] HybridClock initialized: now 1790698207296547 us; error 52 us; skew 500 ppm
I20260929 16:10:07.297062 10416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:07 dist-test-slave-x07d krb5kdc[10403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698207, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:07.300544 10416 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:10:07.302302 10416 webserver.cc:460] Webserver started at http://127.20.121.190:34407/ using document root <none> and password file <none>
I20260929 16:10:07.303058 10416 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:07.303218 10416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:07.303545 10416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:07.307065 10416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "af9b50e04f334bc3b2f1cc498ed1fef4"
format_stamp: "Formatted at 2026-09-29 16:10:07 on dist-test-slave-x07d"
server_key: "b08e8a3dd5e3b5fb7c8d363ba192cc67"
server_key_iv: "c829747a4f84bf37d1ded09216ede451"
server_key_version: "encryptionkey@0"
I20260929 16:10:07.307960 10416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "af9b50e04f334bc3b2f1cc498ed1fef4"
format_stamp: "Formatted at 2026-09-29 16:10:07 on dist-test-slave-x07d"
server_key: "b08e8a3dd5e3b5fb7c8d363ba192cc67"
server_key_iv: "c829747a4f84bf37d1ded09216ede451"
server_key_version: "encryptionkey@0"
I20260929 16:10:07.314569 10416 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260929 16:10:07.318920 10431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:07.320650 10416 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 16:10:07.320948 10416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "af9b50e04f334bc3b2f1cc498ed1fef4"
format_stamp: "Formatted at 2026-09-29 16:10:07 on dist-test-slave-x07d"
server_key: "b08e8a3dd5e3b5fb7c8d363ba192cc67"
server_key_iv: "c829747a4f84bf37d1ded09216ede451"
server_key_version: "encryptionkey@0"
I20260929 16:10:07.321218 10416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:07.359970 10416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:07.364835 10416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:07.365180 10416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:07.386915 10416 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:45243
I20260929 16:10:07.386948 10482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:45243 every 8 connection(s)
I20260929 16:10:07.388563 10416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:10:07.392516 10483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:07.394068 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 10416
I20260929 16:10:07.394266 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:10:07.394797 20966 external_mini_cluster.cc:1428] Setting key 9aa4a017ffc99fd156a71c118bb8e64d
I20260929 16:10:07.408280 10483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4: Bootstrap starting.
I20260929 16:10:07.410111 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:07.396752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46821 (local address 127.20.121.190:45243)
0929 16:10:07.397667 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:07.397684 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:07.397737 (+ 53us) server_negotiation.cc:407] Connection header received
0929 16:10:07.399580 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:07.399606 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:07.400094 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:07.400470 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:07.401254 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:07.402804 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:07.405113 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:07.405937 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:07.406171 (+ 234us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:07.407520 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:07.407528 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:07.407544 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:07.408127 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:07.408213 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:07.408240 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:07.408367 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:07.408737 (+ 370us) server_negotiation.cc:299] Negotiation successful
0929 16:10:07.408998 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":429,"threads_started":1}
I20260929 16:10:07.412237 10483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:07.413592 10483 log.cc:824] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:07.414354 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:10:07.416879 10483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4: No bootstrap required, opened a new log
I20260929 16:10:07.426816 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:07.419207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57917 (local address 127.20.121.190:45243)
0929 16:10:07.419455 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:07.419460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:07.419861 (+ 401us) server_negotiation.cc:407] Connection header received
0929 16:10:07.420057 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:07.420061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:07.420127 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:07.420210 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:07.420911 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:07.421744 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:07.423883 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:07.424328 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:07.424459 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:07.425895 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:07.425900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:07.425904 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:07.425923 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:07.425979 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:07.425995 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:07.426147 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:07.426370 (+ 223us) server_negotiation.cc:299] Negotiation successful
0929 16:10:07.426424 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 16:10:07.428495 10483 raft_consensus.cc:348] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9b50e04f334bc3b2f1cc498ed1fef4" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45243 } }
I20260929 16:10:07.428953 10483 raft_consensus.cc:374] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:10:07.429100 10483 raft_consensus.cc:729] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af9b50e04f334bc3b2f1cc498ed1fef4, State: Initialized, Role: FOLLOWER
I20260929 16:10:07.429806 10483 consensus_queue.cc:260] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9b50e04f334bc3b2f1cc498ed1fef4" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45243 } }
I20260929 16:10:07.430135 10483 raft_consensus.cc:388] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:10:07.430328 10483 raft_consensus.cc:482] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:10:07.430552 10483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:07.433092 10483 raft_consensus.cc:504] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9b50e04f334bc3b2f1cc498ed1fef4" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45243 } }
I20260929 16:10:07.433717 10483 leader_election.cc:302] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af9b50e04f334bc3b2f1cc498ed1fef4; no voters:
I20260929 16:10:07.434567 10483 leader_election.cc:288] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:10:07.434763 10493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:10:07.436925 10483 sys_catalog.cc:564] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:07.438423 10493 raft_consensus.cc:686] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [term 1 LEADER]: Becoming Leader. State: Replica: af9b50e04f334bc3b2f1cc498ed1fef4, State: Running, Role: LEADER
I20260929 16:10:07.439262 10493 consensus_queue.cc:237] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9b50e04f334bc3b2f1cc498ed1fef4" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45243 } }
I20260929 16:10:07.443006 10494 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af9b50e04f334bc3b2f1cc498ed1fef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9b50e04f334bc3b2f1cc498ed1fef4" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45243 } } }
I20260929 16:10:07.442965 10495 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af9b50e04f334bc3b2f1cc498ed1fef4. Latest consensus state: current_term: 1 leader_uuid: "af9b50e04f334bc3b2f1cc498ed1fef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9b50e04f334bc3b2f1cc498ed1fef4" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 45243 } } }
I20260929 16:10:07.443538 10494 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:07.443572 10495 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:07.445366 10505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:10:07.451354 10505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:10:07.462518 10505 catalog_manager.cc:1347] Generated new cluster ID: 4f7ee01f557c45adb08cf83335d14cd3
I20260929 16:10:07.462675 10505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:10:07.499526 10505 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:10:07.501616 10505 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:10:07.509500 10505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af9b50e04f334bc3b2f1cc498ed1fef4: Generated new TSK 0
I20260929 16:10:07.510720 10505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:10:07.523669 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10416
W20260929 16:10:07.525401 10478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46821
W20260929 16:10:07.526367 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10416: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:10:07.526551 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 10416
W20260929 16:10:07.545444 10401 connection.cc:535] client connection to 127.20.121.190:45243 recv error: Network error: failed to read from TLS socket (remote: 127.20.121.190:45243): Connection reset by peer (error 104)
2026-09-29T16:10:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2105 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:10:07 dist-test-slave-x07d krb5kdc[10514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:10:07 dist-test-slave-x07d krb5kdc[10514](info): set up 2 sockets
Sep 29 16:10:07 dist-test-slave-x07d krb5kdc[10514](info): commencing operation
krb5kdc: starting...
W20260929 16:10:08.714812 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.005s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:10:08 dist-test-slave-x07d krb5kdc[10514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:10:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:10:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:10:08.999490 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34601
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:34601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:10:09.347965 10527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:09.348313 10527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:09.348497 10527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:09.357175 10527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:09.357334 10527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:09.357431 10527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:09.357565 10527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:09.357694 10527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:09.368294 10527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:34601
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:34601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:09.369766 10527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:09.371253 10527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:09.385012 10535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:09.384943 10532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:09.384964 10533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:09.385552 10527 server_base.cc:1034] running on GCE node
I20260929 16:10:09.387306 10527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:09.389819 10527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:09.391050 10527 hybrid_clock.cc:648] HybridClock initialized: now 1790698209390916 us; error 121 us; skew 500 ppm
I20260929 16:10:09.391553 10527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:09 dist-test-slave-x07d krb5kdc[10514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698209, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:09.395617 10527 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:10:09.397430 10527 webserver.cc:460] Webserver started at http://127.20.121.190:42391/ using document root <none> and password file <none>
I20260929 16:10:09.398249 10527 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:09.398412 10527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:09.398734 10527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:09.401953 10527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "c50ee77fc2a5472f8a555c6743ac2d6a"
format_stamp: "Formatted at 2026-09-29 16:10:09 on dist-test-slave-x07d"
server_key: "537ca25a623c6bb3c8d9f7b828e5f0a6"
server_key_iv: "63f34247d2d75241d1033dd82c7bb87d"
server_key_version: "encryptionkey@0"
I20260929 16:10:09.402882 10527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "c50ee77fc2a5472f8a555c6743ac2d6a"
format_stamp: "Formatted at 2026-09-29 16:10:09 on dist-test-slave-x07d"
server_key: "537ca25a623c6bb3c8d9f7b828e5f0a6"
server_key_iv: "63f34247d2d75241d1033dd82c7bb87d"
server_key_version: "encryptionkey@0"
I20260929 16:10:09.408648 10527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:10:09.412626 10542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:09.414222 10527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 16:10:09.414433 10527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "c50ee77fc2a5472f8a555c6743ac2d6a"
format_stamp: "Formatted at 2026-09-29 16:10:09 on dist-test-slave-x07d"
server_key: "537ca25a623c6bb3c8d9f7b828e5f0a6"
server_key_iv: "63f34247d2d75241d1033dd82c7bb87d"
server_key_version: "encryptionkey@0"
I20260929 16:10:09.414660 10527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:09.421242 10527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:09.425587 10527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:09.425905 10527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:09.445663 10527 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:34601
I20260929 16:10:09.445696 10593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:34601 every 8 connection(s)
I20260929 16:10:09.447165 10527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:10:09.447670 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 10527
I20260929 16:10:09.447841 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:10:09.448438 20966 external_mini_cluster.cc:1428] Setting key 7956887048164199e2f3dd9202cfda8c
I20260929 16:10:09.452005 10594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:09.456842 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:09.450488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57785 (local address 127.20.121.190:34601)
0929 16:10:09.451296 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:09.451308 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:09.451355 (+ 47us) server_negotiation.cc:407] Connection header received
0929 16:10:09.453171 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:09.453194 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:09.453535 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:09.453957 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:09.455143 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:09.455148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:09.455161 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:09.455176 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:09.455250 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:09.455269 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:09.455509 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:09.455653 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 16:10:09.455861 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":382,"threads_started":1}
I20260929 16:10:09.460863 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:10:09.464327 10594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a: Bootstrap starting.
I20260929 16:10:09.468446 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:09.465487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60847 (local address 127.20.121.190:34601)
0929 16:10:09.465651 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:09.465655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:09.466257 (+ 602us) server_negotiation.cc:407] Connection header received
0929 16:10:09.466477 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:09.466485 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:09.466566 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:09.466690 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:09.467702 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:09.467706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:09.467709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:09.467724 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:09.467766 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:09.467781 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:09.467887 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:09.468063 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 16:10:09.468112 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 16:10:09.468632 10594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:09.470124 10594 log.cc:824] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:09.473075 10594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a: No bootstrap required, opened a new log
I20260929 16:10:09.481249 10594 raft_consensus.cc:348] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee77fc2a5472f8a555c6743ac2d6a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34601 } }
I20260929 16:10:09.481778 10594 raft_consensus.cc:374] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:10:09.481909 10594 raft_consensus.cc:729] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c50ee77fc2a5472f8a555c6743ac2d6a, State: Initialized, Role: FOLLOWER
I20260929 16:10:09.482640 10594 consensus_queue.cc:260] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee77fc2a5472f8a555c6743ac2d6a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34601 } }
I20260929 16:10:09.482910 10594 raft_consensus.cc:388] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:10:09.483091 10594 raft_consensus.cc:482] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:10:09.483294 10594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:09.485788 10594 raft_consensus.cc:504] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee77fc2a5472f8a555c6743ac2d6a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34601 } }
I20260929 16:10:09.486452 10594 leader_election.cc:302] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c50ee77fc2a5472f8a555c6743ac2d6a; no voters:
I20260929 16:10:09.487171 10594 leader_election.cc:288] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:10:09.487293 10604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:10:09.488035 10604 raft_consensus.cc:686] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [term 1 LEADER]: Becoming Leader. State: Replica: c50ee77fc2a5472f8a555c6743ac2d6a, State: Running, Role: LEADER
I20260929 16:10:09.488695 10604 consensus_queue.cc:237] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee77fc2a5472f8a555c6743ac2d6a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34601 } }
I20260929 16:10:09.489578 10594 sys_catalog.cc:564] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:09.492116 10606 sys_catalog.cc:455] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c50ee77fc2a5472f8a555c6743ac2d6a. Latest consensus state: current_term: 1 leader_uuid: "c50ee77fc2a5472f8a555c6743ac2d6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee77fc2a5472f8a555c6743ac2d6a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34601 } } }
I20260929 16:10:09.491983 10605 sys_catalog.cc:455] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c50ee77fc2a5472f8a555c6743ac2d6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c50ee77fc2a5472f8a555c6743ac2d6a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 34601 } } }
I20260929 16:10:09.492503 10606 sys_catalog.cc:458] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:09.492548 10605 sys_catalog.cc:458] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:09.494858 10613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:10:09.501124 10613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:10:09.511307 10613 catalog_manager.cc:1347] Generated new cluster ID: 003836a45c6e4d5aafb59601492bd56f
I20260929 16:10:09.511445 10613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:10:09.525929 10613 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:10:09.526432 10613 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:10:09.532972 10613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c50ee77fc2a5472f8a555c6743ac2d6a: Generated new TSK 0
I20260929 16:10:09.533838 10613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:10:09.547076 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10527
W20260929 16:10:09.548218 10589 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57785
W20260929 16:10:09.548990 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10527: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:10:09.549156 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 10527
2026-09-29T16:10:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2021 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:10:09 dist-test-slave-x07d krb5kdc[10625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:10:09 dist-test-slave-x07d krb5kdc[10625](info): set up 2 sockets
Sep 29 16:10:09 dist-test-slave-x07d krb5kdc[10625](info): commencing operation
krb5kdc: starting...
W20260929 16:10:10.730876 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:10:10 dist-test-slave-x07d krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:10:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:10:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:10:11.019203 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37095
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:37095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:10:11.361958 10638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:11.362318 10638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:11.362500 10638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:11.370150 10638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:11.370280 10638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:11.370388 10638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:11.370504 10638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:11.370586 10638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:11.381183 10638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:37095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:11.383008 10638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:11.384531 10638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:11.397354 10643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:11.397428 10644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:11.397305 10646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:11.398367 10638 server_base.cc:1034] running on GCE node
I20260929 16:10:11.399236 10638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:11.400935 10638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:11.402204 10638 hybrid_clock.cc:648] HybridClock initialized: now 1790698211402144 us; error 58 us; skew 500 ppm
I20260929 16:10:11.402612 10638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:11 dist-test-slave-x07d krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698211, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:11.405907 10638 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:10:11.407752 10638 webserver.cc:460] Webserver started at http://127.20.121.190:35021/ using document root <none> and password file <none>
I20260929 16:10:11.408514 10638 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:11.408670 10638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:11.408995 10638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:11.412253 10638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "725f622ad55448b5880bfb9109d75bd0"
format_stamp: "Formatted at 2026-09-29 16:10:11 on dist-test-slave-x07d"
server_key: "e8b518e4c4880aa9f221d923d6b3ee81"
server_key_iv: "d240c76e45b1fd18af8b201b95cbbd2c"
server_key_version: "encryptionkey@0"
I20260929 16:10:11.413130 10638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "725f622ad55448b5880bfb9109d75bd0"
format_stamp: "Formatted at 2026-09-29 16:10:11 on dist-test-slave-x07d"
server_key: "e8b518e4c4880aa9f221d923d6b3ee81"
server_key_iv: "d240c76e45b1fd18af8b201b95cbbd2c"
server_key_version: "encryptionkey@0"
I20260929 16:10:11.419211 10638 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260929 16:10:11.423583 10653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:11.425284 10638 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260929 16:10:11.425549 10638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "725f622ad55448b5880bfb9109d75bd0"
format_stamp: "Formatted at 2026-09-29 16:10:11 on dist-test-slave-x07d"
server_key: "e8b518e4c4880aa9f221d923d6b3ee81"
server_key_iv: "d240c76e45b1fd18af8b201b95cbbd2c"
server_key_version: "encryptionkey@0"
I20260929 16:10:11.425812 10638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:11.445971 10638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:11.450922 10638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:11.451285 10638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:11.471478 10638 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:37095
I20260929 16:10:11.471501 10704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:37095 every 8 connection(s)
I20260929 16:10:11.472980 10638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:10:11.476475 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 10638
I20260929 16:10:11.476660 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:10:11.477102 20966 external_mini_cluster.cc:1428] Setting key c29f32ceeea22083d80bf309fc99c4ab
I20260929 16:10:11.477196 10705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:11.492803 10705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0: Bootstrap starting.
Sep 29 16:10:11 dist-test-slave-x07d krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:10:11.496805 10705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:11.498401 10705 log.cc:824] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:11.502422 10705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0: No bootstrap required, opened a new log
I20260929 16:10:11.504642 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:11.479106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59395 (local address 127.20.121.190:37095)
0929 16:10:11.479951 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:11.479967 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:11.480029 (+ 62us) server_negotiation.cc:407] Connection header received
0929 16:10:11.482552 (+ 2523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:11.482586 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:11.483233 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:11.483748 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:11.485260 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:11.487297 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:11.489082 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:11.490349 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:11.490577 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:11.495000 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:11.495012 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:11.495033 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:11.495096 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:11.499454 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:11.500448 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:11.500457 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:11.500466 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:11.500699 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:11.501118 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:11.501122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:11.501125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:11.501901 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:11.502232 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:11.502658 (+ 426us) server_negotiation.cc:299] Negotiation successful
0929 16:10:11.503233 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":363,"threads_started":1}
I20260929 16:10:11.508925 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:10:11.512120 10705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725f622ad55448b5880bfb9109d75bd0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37095 } }
I20260929 16:10:11.512593 10705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:10:11.512705 10705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 725f622ad55448b5880bfb9109d75bd0, State: Initialized, Role: FOLLOWER
I20260929 16:10:11.513398 10705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725f622ad55448b5880bfb9109d75bd0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37095 } }
I20260929 16:10:11.513763 10705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:10:11.513926 10705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:10:11.514174 10705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:11.516999 10705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725f622ad55448b5880bfb9109d75bd0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37095 } }
I20260929 16:10:11.517663 10705 leader_election.cc:302] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 725f622ad55448b5880bfb9109d75bd0; no voters:
I20260929 16:10:11.518384 10705 leader_election.cc:288] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:10:11.518538 10715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:10:11.520768 10705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:11.521828 10715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [term 1 LEADER]: Becoming Leader. State: Replica: 725f622ad55448b5880bfb9109d75bd0, State: Running, Role: LEADER
I20260929 16:10:11.522714 10715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725f622ad55448b5880bfb9109d75bd0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37095 } }
I20260929 16:10:11.527097 10717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 725f622ad55448b5880bfb9109d75bd0. Latest consensus state: current_term: 1 leader_uuid: "725f622ad55448b5880bfb9109d75bd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725f622ad55448b5880bfb9109d75bd0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37095 } } }
I20260929 16:10:11.527568 10717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:11.528177 10716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "725f622ad55448b5880bfb9109d75bd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725f622ad55448b5880bfb9109d75bd0" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37095 } } }
I20260929 16:10:11.528504 10716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:11.529546 10729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:10:11.532589 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:11.513685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56151 (local address 127.20.121.190:37095)
0929 16:10:11.513905 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:11.513916 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:11.514438 (+ 522us) server_negotiation.cc:407] Connection header received
0929 16:10:11.514583 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:11.514591 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:11.514679 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:11.514800 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:11.516215 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:11.517314 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:11.519189 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:11.520229 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:11.520394 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:11.524228 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:11.524233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:11.524237 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:11.524274 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:11.528632 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:11.529818 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:11.529823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:11.529826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:11.529981 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:11.530520 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:11.530525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:11.530528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:11.530963 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:11.531235 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:11.531705 (+ 470us) server_negotiation.cc:299] Negotiation successful
0929 16:10:11.532018 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 16:10:11.536425 10729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:10:11.545994 10729 catalog_manager.cc:1347] Generated new cluster ID: 9fa19e2e931640bc9174088499afa7e6
I20260929 16:10:11.546217 10729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:10:11.566665 10729 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:10:11.568197 10729 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:10:11.579097 10729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 725f622ad55448b5880bfb9109d75bd0: Generated new TSK 0
I20260929 16:10:11.580209 10729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:10:11.596282 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10638
I20260929 16:10:11.597649 10700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:10:11.693468 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 10638
2026-09-29T16:10:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2143 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:10:11 dist-test-slave-x07d krb5kdc[10737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:10:11 dist-test-slave-x07d krb5kdc[10737](info): set up 2 sockets
Sep 29 16:10:11 dist-test-slave-x07d krb5kdc[10737](info): commencing operation
krb5kdc: starting...
W20260929 16:10:12.834544 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:10:13 dist-test-slave-x07d krb5kdc[10737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:10:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:10:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:10:13.118755 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:39491
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:45339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:39491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:10:13.468338 10750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:13.468670 10750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:13.468853 10750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:13.476418 10750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:13.476549 10750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:13.476652 10750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:13.476760 10750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:13.476838 10750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:13.487427 10750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:45339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:39491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:13.488884 10750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:13.490350 10750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:13.502957 10756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:13.503019 10758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:13.503019 10755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:13.504308 10750 server_base.cc:1034] running on GCE node
I20260929 16:10:13.505240 10750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:13.507025 10750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:13.508275 10750 hybrid_clock.cc:648] HybridClock initialized: now 1790698213508225 us; error 46 us; skew 500 ppm
I20260929 16:10:13.508657 10750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:13 dist-test-slave-x07d krb5kdc[10737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698213, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:13.516894 10750 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:10:13.519037 10750 webserver.cc:460] Webserver started at http://127.20.121.190:32955/ using document root <none> and password file <none>
I20260929 16:10:13.519850 10750 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:13.520009 10750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:13.520354 10750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:13.523798 10750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "daeee9648e79430196c7eebcf0d792c1"
format_stamp: "Formatted at 2026-09-29 16:10:13 on dist-test-slave-x07d"
server_key: "3751c1f1bb8212ac9fa3ee215331b1ec"
server_key_iv: "fa940a709706b771ad887dcfc8fe60ba"
server_key_version: "encryptionkey@0"
I20260929 16:10:13.524724 10750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "daeee9648e79430196c7eebcf0d792c1"
format_stamp: "Formatted at 2026-09-29 16:10:13 on dist-test-slave-x07d"
server_key: "3751c1f1bb8212ac9fa3ee215331b1ec"
server_key_iv: "fa940a709706b771ad887dcfc8fe60ba"
server_key_version: "encryptionkey@0"
I20260929 16:10:13.530558 10750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260929 16:10:13.534339 10765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:13.535822 10750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:10:13.536060 10750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "daeee9648e79430196c7eebcf0d792c1"
format_stamp: "Formatted at 2026-09-29 16:10:13 on dist-test-slave-x07d"
server_key: "3751c1f1bb8212ac9fa3ee215331b1ec"
server_key_iv: "fa940a709706b771ad887dcfc8fe60ba"
server_key_version: "encryptionkey@0"
I20260929 16:10:13.536278 10750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:13.568789 10750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:13.573455 10750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:13.573791 10750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:13.592947 10750 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:39491
I20260929 16:10:13.592993 10816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:39491 every 8 connection(s)
I20260929 16:10:13.594514 10750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:10:13.596124 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 10750
I20260929 16:10:13.596326 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:10:13.596800 20966 external_mini_cluster.cc:1428] Setting key 1d7bebdb91a83886b589c40b791b9bc6
I20260929 16:10:13.599292 10817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:13.612857 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:13.598621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33417 (local address 127.20.121.190:39491)
0929 16:10:13.599465 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:13.599480 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:13.599533 (+ 53us) server_negotiation.cc:407] Connection header received
0929 16:10:13.601918 (+ 2385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:13.601949 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:13.602528 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:13.603030 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:13.603615 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:13.605693 (+ 2078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:13.607187 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:13.608250 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:13.608458 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:13.609641 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:13.609652 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:13.609701 (+ 49us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:13.610541 (+ 840us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:13.610657 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:13.610691 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:13.610827 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:13.611091 (+ 264us) server_negotiation.cc:299] Negotiation successful
0929 16:10:13.611408 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":334,"threads_started":1}
I20260929 16:10:13.614540 10817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1: Bootstrap starting.
I20260929 16:10:13.617004 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:10:13.618172 10817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:13.619252 10817 log.cc:824] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:13.622846 10817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1: No bootstrap required, opened a new log
I20260929 16:10:13.628647 10820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:13.621650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52045 (local address 127.20.121.190:39491)
0929 16:10:13.621887 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:13.621897 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:13.622395 (+ 498us) server_negotiation.cc:407] Connection header received
0929 16:10:13.622550 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:13.622556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:13.622632 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:13.622745 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:13.623380 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:13.624466 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:13.625788 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:13.626542 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:13.626690 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:13.627729 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:13.627735 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:13.627740 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:13.627762 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:13.627821 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:13.627857 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:13.627964 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:13.628152 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 16:10:13.628262 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 16:10:13.632324 10817 raft_consensus.cc:348] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daeee9648e79430196c7eebcf0d792c1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 39491 } }
I20260929 16:10:13.632762 10817 raft_consensus.cc:374] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:10:13.632885 10817 raft_consensus.cc:729] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daeee9648e79430196c7eebcf0d792c1, State: Initialized, Role: FOLLOWER
I20260929 16:10:13.633535 10817 consensus_queue.cc:260] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daeee9648e79430196c7eebcf0d792c1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 39491 } }
I20260929 16:10:13.633771 10817 raft_consensus.cc:388] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:10:13.633912 10817 raft_consensus.cc:482] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:10:13.634104 10817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:13.636358 10817 raft_consensus.cc:504] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daeee9648e79430196c7eebcf0d792c1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 39491 } }
I20260929 16:10:13.636924 10817 leader_election.cc:302] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daeee9648e79430196c7eebcf0d792c1; no voters:
I20260929 16:10:13.637511 10817 leader_election.cc:288] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:10:13.637696 10827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:10:13.638397 10827 raft_consensus.cc:686] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [term 1 LEADER]: Becoming Leader. State: Replica: daeee9648e79430196c7eebcf0d792c1, State: Running, Role: LEADER
I20260929 16:10:13.639031 10827 consensus_queue.cc:237] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daeee9648e79430196c7eebcf0d792c1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 39491 } }
I20260929 16:10:13.640074 10817 sys_catalog.cc:564] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:13.642045 10829 sys_catalog.cc:455] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader daeee9648e79430196c7eebcf0d792c1. Latest consensus state: current_term: 1 leader_uuid: "daeee9648e79430196c7eebcf0d792c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daeee9648e79430196c7eebcf0d792c1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 39491 } } }
I20260929 16:10:13.642148 10828 sys_catalog.cc:455] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daeee9648e79430196c7eebcf0d792c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daeee9648e79430196c7eebcf0d792c1" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 39491 } } }
I20260929 16:10:13.642520 10829 sys_catalog.cc:458] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:13.642684 10828 sys_catalog.cc:458] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:13.643644 10834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:10:13.650575 10834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:10:13.661147 10834 catalog_manager.cc:1347] Generated new cluster ID: c7ca3b2297b6452895cbc52d679a2c6f
I20260929 16:10:13.661288 10834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:10:13.682080 10834 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:10:13.683640 10834 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:10:13.691390 10834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P daeee9648e79430196c7eebcf0d792c1: Generated new TSK 0
I20260929 16:10:13.692590 10834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:10:13.699756 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10750
W20260929 16:10:13.701704 10812 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33417
W20260929 16:10:13.702477 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10750: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:10:13.702642 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 10750
2026-09-29T16:10:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2008 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:10:13 dist-test-slave-x07d krb5kdc[10848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:10:13 dist-test-slave-x07d krb5kdc[10848](info): set up 2 sockets
Sep 29 16:10:13 dist-test-slave-x07d krb5kdc[10848](info): commencing operation
krb5kdc: starting...
W20260929 16:10:14.822696 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:10:14 dist-test-slave-x07d krb5kdc[10848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:10:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:10:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:10:15.106246 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37249
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:46643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:37249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:10:15.461498 10861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:15.461889 10861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:15.462121 10861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:15.469640 10861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:15.469835 10861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:15.469945 10861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:15.470064 10861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:15.470134 10861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:15.480604 10861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:46643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:37249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:37249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:15.481989 10861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:15.483440 10861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:15.496685 10866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:15.496747 10867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:15.496727 10869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:15.497731 10861 server_base.cc:1034] running on GCE node
I20260929 16:10:15.498660 10861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:15.500289 10861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:15.501528 10861 hybrid_clock.cc:648] HybridClock initialized: now 1790698215501472 us; error 50 us; skew 500 ppm
I20260929 16:10:15.501981 10861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:15 dist-test-slave-x07d krb5kdc[10848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698215, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:15.505609 10861 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:10:15.507493 10861 webserver.cc:460] Webserver started at http://127.20.121.190:37487/ using document root <none> and password file <none>
I20260929 16:10:15.508291 10861 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:15.508450 10861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:15.508803 10861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:15.512133 10861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242"
format_stamp: "Formatted at 2026-09-29 16:10:15 on dist-test-slave-x07d"
server_key: "1bdee15e490d276a4c18f324490bcc78"
server_key_iv: "51e043d39205605f5e733583197bfd0d"
server_key_version: "encryptionkey@0"
I20260929 16:10:15.513033 10861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242"
format_stamp: "Formatted at 2026-09-29 16:10:15 on dist-test-slave-x07d"
server_key: "1bdee15e490d276a4c18f324490bcc78"
server_key_iv: "51e043d39205605f5e733583197bfd0d"
server_key_version: "encryptionkey@0"
I20260929 16:10:15.518910 10861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 16:10:15.522940 10876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:15.524569 10861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:10:15.524803 10861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242"
format_stamp: "Formatted at 2026-09-29 16:10:15 on dist-test-slave-x07d"
server_key: "1bdee15e490d276a4c18f324490bcc78"
server_key_iv: "51e043d39205605f5e733583197bfd0d"
server_key_version: "encryptionkey@0"
I20260929 16:10:15.525040 10861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:15.531906 10861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:15.536544 10861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:15.536881 10861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:15.557405 10861 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:37249
I20260929 16:10:15.557442 10927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:37249 every 8 connection(s)
I20260929 16:10:15.559008 10861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:10:15.563225 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 10861
I20260929 16:10:15.563436 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:10:15.563560 10928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:15.563957 20966 external_mini_cluster.cc:1428] Setting key 31f4cb7463270d406632d90e6321e652
I20260929 16:10:15.573110 10931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:15.566060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45845 (local address 127.20.121.190:37249)
0929 16:10:15.567021 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:15.567033 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:15.567077 (+ 44us) server_negotiation.cc:407] Connection header received
0929 16:10:15.568962 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:15.568995 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:15.569473 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:15.569953 (+ 480us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:15.571239 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:15.571247 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:15.571261 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:15.571281 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:15.571365 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:15.571388 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:15.571602 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:15.571718 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 16:10:15.572071 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":437,"threads_started":1}
I20260929 16:10:15.577489 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:10:15.579149 10928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242: Bootstrap starting.
I20260929 16:10:15.583979 10928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:15.585633 10928 log.cc:824] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:15.586565 10931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:15.583227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33693 (local address 127.20.121.190:37249)
0929 16:10:15.583475 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:15.583482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:15.583912 (+ 430us) server_negotiation.cc:407] Connection header received
0929 16:10:15.584152 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:15.584158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:15.584251 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:15.584381 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:15.585557 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:15.585561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:15.585564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 16:10:15.585581 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:15.585637 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 16:10:15.585653 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:15.585900 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:15.586117 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 16:10:15.586184 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 16:10:15.589469 10928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242: No bootstrap required, opened a new log
I20260929 16:10:15.597474 10928 raft_consensus.cc:348] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37249 } }
I20260929 16:10:15.597966 10928 raft_consensus.cc:374] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 16:10:15.598109 10928 raft_consensus.cc:729] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c906a1ad81ed49f0bc6bc17f1a5bd242, State: Initialized, Role: FOLLOWER
I20260929 16:10:15.598764 10928 consensus_queue.cc:260] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37249 } }
I20260929 16:10:15.599051 10928 raft_consensus.cc:388] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 16:10:15.599228 10928 raft_consensus.cc:482] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 16:10:15.599447 10928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:15.601807 10928 raft_consensus.cc:504] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37249 } }
I20260929 16:10:15.602452 10928 leader_election.cc:302] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c906a1ad81ed49f0bc6bc17f1a5bd242; no voters:
I20260929 16:10:15.603255 10928 leader_election.cc:288] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 16:10:15.603443 10938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:10:15.604403 10938 raft_consensus.cc:686] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [term 1 LEADER]: Becoming Leader. State: Replica: c906a1ad81ed49f0bc6bc17f1a5bd242, State: Running, Role: LEADER
I20260929 16:10:15.605032 10938 consensus_queue.cc:237] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37249 } }
I20260929 16:10:15.605841 10928 sys_catalog.cc:564] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:15.608985 10939 sys_catalog.cc:455] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37249 } } }
I20260929 16:10:15.609532 10939 sys_catalog.cc:458] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:15.608971 10940 sys_catalog.cc:455] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c906a1ad81ed49f0bc6bc17f1a5bd242. Latest consensus state: current_term: 1 leader_uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c906a1ad81ed49f0bc6bc17f1a5bd242" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 37249 } } }
I20260929 16:10:15.610520 10940 sys_catalog.cc:458] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:15.611207 10947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:10:15.617532 10947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:10:15.630287 10947 catalog_manager.cc:1347] Generated new cluster ID: e9ba195881504ba8b0990f20cf934020
I20260929 16:10:15.630427 10947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:10:15.652226 10947 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:10:15.652849 10947 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:10:15.663388 10947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c906a1ad81ed49f0bc6bc17f1a5bd242: Generated new TSK 0
I20260929 16:10:15.664698 10947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:10:15.682083 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10861
W20260929 16:10:15.683266 10923 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45845
W20260929 16:10:15.684047 20966 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10861: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 16:10:15.684209 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 10861
2026-09-29T16:10:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1981 ms)
[----------] 18 tests from AuthTokenIssuingTest (37139 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:10:15 dist-test-slave-x07d krb5kdc[10959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:10:15 dist-test-slave-x07d krb5kdc[10959](info): set up 2 sockets
Sep 29 16:10:15 dist-test-slave-x07d krb5kdc[10959](info): commencing operation
krb5kdc: starting...
W20260929 16:10:16.838976 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:10:17 dist-test-slave-x07d krb5kdc[10959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:10:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:10:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:10:17.118701 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33931
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:37997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:33931,127.20.121.189:37581,127.20.121.188:44743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:10:17.462827 10972 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 16:10:17.463289 10972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:17.463505 10972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:17.463636 10972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:17.471271 10972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:17.471405 10972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:17.471505 10972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:17.471588 10972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:17.471695 10972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:17.481973 10972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:37997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:33931,127.20.121.189:37581,127.20.121.188:44743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:33931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:17.483458 10972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:17.484841 10972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:17.497655 10980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:17.497653 10978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:17.497859 10977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:17.499009 10972 server_base.cc:1034] running on GCE node
I20260929 16:10:17.499825 10972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:17.501410 10972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:17.502661 10972 hybrid_clock.cc:648] HybridClock initialized: now 1790698217502604 us; error 40 us; skew 500 ppm
I20260929 16:10:17.503018 10972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:17 dist-test-slave-x07d krb5kdc[10959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698217, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:17.506338 10972 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:10:17.507879 10972 webserver.cc:460] Webserver started at http://127.20.121.190:36139/ using document root <none> and password file <none>
I20260929 16:10:17.508587 10972 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:17.508715 10972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:17.509047 10972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:17.512296 10972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "c1c63c01777f468faea535e4c6969e50"
format_stamp: "Formatted at 2026-09-29 16:10:17 on dist-test-slave-x07d"
server_key: "64a9e9bc8427ed86f5554622964c60f6"
server_key_iv: "43b204b9605b97c3fbca5f9ca1ca7b34"
server_key_version: "encryptionkey@0"
I20260929 16:10:17.513147 10972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "c1c63c01777f468faea535e4c6969e50"
format_stamp: "Formatted at 2026-09-29 16:10:17 on dist-test-slave-x07d"
server_key: "64a9e9bc8427ed86f5554622964c60f6"
server_key_iv: "43b204b9605b97c3fbca5f9ca1ca7b34"
server_key_version: "encryptionkey@0"
I20260929 16:10:17.518533 10972 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 16:10:17.522256 10987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:17.523635 10972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 16:10:17.523826 10972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "c1c63c01777f468faea535e4c6969e50"
format_stamp: "Formatted at 2026-09-29 16:10:17 on dist-test-slave-x07d"
server_key: "64a9e9bc8427ed86f5554622964c60f6"
server_key_iv: "43b204b9605b97c3fbca5f9ca1ca7b34"
server_key_version: "encryptionkey@0"
I20260929 16:10:17.524068 10972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:17.538364 10972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:17.543074 10972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:17.543381 10972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:17.562613 10972 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:33931
I20260929 16:10:17.562618 11038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:33931 every 8 connection(s)
I20260929 16:10:17.564093 10972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:10:17.565758 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 10972
I20260929 16:10:17.566058 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:10:17.566879 20966 external_mini_cluster.cc:1428] Setting key 4e83c396ae0dc7acdf7f6c08bc664adc
I20260929 16:10:17.569010 11039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:17.579515 11039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.121.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.189@KRBTEST.COM" created.
Sep 29 16:10:17 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698217, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:10:17.608157 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.583059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51552 (local address 127.20.121.190:33931)
0929 16:10:17.584825 (+ 1766us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:17.584835 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:17.584891 (+ 56us) server_negotiation.cc:407] Connection header received
0929 16:10:17.586726 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:17.586737 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:17.587083 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:17.587323 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:17.588845 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:17.590957 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:17.592630 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:17.593738 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:17.593995 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:17.600470 (+ 6475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:17.600480 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:17.600494 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:17.600528 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:17.603561 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:17.604347 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:17.604354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:17.604357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:17.604507 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:17.604906 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:17.604914 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:17.604917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:17.605463 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:17.605677 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:17.606238 (+ 561us) server_negotiation.cc:299] Negotiation successful
0929 16:10:17.606614 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1538,"spinlock_wait_cycles":59648,"thread_start_us":1308,"threads_started":1}
I20260929 16:10:17.608192 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.583067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33931 (local address 127.0.0.1:51552)
0929 16:10:17.584228 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.584294 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:17.586301 (+ 2007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:17.587407 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:17.587414 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:17.588185 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:17.588581 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:17.588605 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:17.591169 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:17.591174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:17.592453 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:17.592467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:17.593997 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:17.594002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:17.594195 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:17.594930 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:17.594956 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:17.600191 (+ 5235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:17.603720 (+ 3529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:17.603735 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:17.603749 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:17.604185 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:17.604622 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:17.604630 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:17.604634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:17.604796 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:17.605594 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:17.605607 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:17.605824 (+ 217us) client_negotiation.cc:769] Sending connection context
0929 16:10:17.606118 (+ 294us) client_negotiation.cc:240] Negotiation successful
0929 16:10:17.606736 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":141,"thread_start_us":410,"threads_started":1}
I20260929 16:10:17.611409 11039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:10:17.612636 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.612220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37650)
0929 16:10:17.612396 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.612510 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 16:10:17.612924 10988 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
I20260929 16:10:17.614699 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.614316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37654)
0929 16:10:17.614468 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.614546 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 16:10:17.614904 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:37581: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab.
Entry for principal kudu/127.20.121.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab.
I20260929 16:10:17.633817 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.189:37581
--webserver_interface=127.20.121.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab
--principal=kudu/127.20.121.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:37997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:33931,127.20.121.189:37581,127.20.121.188:44743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:10:17.664242 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } attempt: 1
I20260929 16:10:17.665525 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.665076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37666)
0929 16:10:17.665270 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.665396 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 16:10:17.666543 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.666213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37668)
0929 16:10:17.666350 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.666420 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 16:10:17.666724 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:37581: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
I20260929 16:10:17.735015 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } attempt: 2
I20260929 16:10:17.736301 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.735791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37680)
0929 16:10:17.736063 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.736158 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260929 16:10:17.737321 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.736952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37696)
0929 16:10:17.737125 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.737193 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 16:10:17.737506 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:37581: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
I20260929 16:10:17.828862 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } attempt: 3
I20260929 16:10:17.830291 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.829751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37710)
0929 16:10:17.829966 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.830125 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260929 16:10:17.831373 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.830964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37716)
0929 16:10:17.831119 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.831209 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 16:10:17.831562 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:37581: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
I20260929 16:10:17.974838 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } attempt: 4
I20260929 16:10:17.976044 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.975509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37718)
0929 16:10:17.975815 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.975905 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260929 16:10:17.976589 11046 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 16:10:17.976954 11046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:17.977075 11046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260929 16:10:17.977092 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:17.976683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37730)
0929 16:10:17.976828 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:17.976947 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 16:10:17.977224 11046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:17.977290 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:37581: Network error: Client connection negotiation failed: client connection to 127.20.121.189:37581: connect: Connection refused (error 111)
W20260929 16:10:17.984814 11046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:17.984920 11046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:17.985011 11046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:17.985109 11046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:17.985219 11046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:17.995566 11046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:37997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:33931,127.20.121.189:37581,127.20.121.188:44743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab
--principal=kudu/127.20.121.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.189:37581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.121.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:17.996948 11046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:17.998411 11046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:18.010828 11051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:18.010800 11052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:18.011114 11054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:18.011713 11046 server_base.cc:1034] running on GCE node
I20260929 16:10:18.012531 11046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:18.014118 11046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:18.015358 11046 hybrid_clock.cc:648] HybridClock initialized: now 1790698218015277 us; error 55 us; skew 500 ppm
I20260929 16:10:18.015724 11046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698218, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:18.023566 11046 init.cc:377] Logged in from keytab as kudu/127.20.121.189@KRBTEST.COM (short username kudu)
I20260929 16:10:18.025279 11046 webserver.cc:460] Webserver started at http://127.20.121.189:39467/ using document root <none> and password file <none>
I20260929 16:10:18.026086 11046 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:18.026204 11046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:18.026552 11046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:18.029870 11046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/data/instance:
uuid: "8c9186e44d974aea833e1be239bb745d"
format_stamp: "Formatted at 2026-09-29 16:10:18 on dist-test-slave-x07d"
server_key: "7fdb0e371705ba03ca70d2851a996686"
server_key_iv: "ca300ed732966b8b004602768ac867e3"
server_key_version: "encryptionkey@0"
I20260929 16:10:18.030817 11046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/wal/instance:
uuid: "8c9186e44d974aea833e1be239bb745d"
format_stamp: "Formatted at 2026-09-29 16:10:18 on dist-test-slave-x07d"
server_key: "7fdb0e371705ba03ca70d2851a996686"
server_key_iv: "ca300ed732966b8b004602768ac867e3"
server_key_version: "encryptionkey@0"
I20260929 16:10:18.036523 11046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:10:18.040364 11061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:18.041814 11046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:10:18.042021 11046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/wal
uuid: "8c9186e44d974aea833e1be239bb745d"
format_stamp: "Formatted at 2026-09-29 16:10:18 on dist-test-slave-x07d"
server_key: "7fdb0e371705ba03ca70d2851a996686"
server_key_iv: "ca300ed732966b8b004602768ac867e3"
server_key_version: "encryptionkey@0"
I20260929 16:10:18.042292 11046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:18.060526 11046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:18.065059 11046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:18.065353 11046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:18.085279 11046 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.189:37581
I20260929 16:10:18.085301 11112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.189:37581 every 8 connection(s)
I20260929 16:10:18.086763 11046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
I20260929 16:10:18.090245 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 11046
I20260929 16:10:18.090459 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-1/wal/instance
I20260929 16:10:18.091010 20966 external_mini_cluster.cc:1428] Setting key 55f1241d3d2f9029e05af8af30b34cac
I20260929 16:10:18.091089 11113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:18.101620 11113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698218, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.188@KRBTEST.COM" created.
I20260929 16:10:18.126637 11043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.104525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.20.121.190:33931)
0929 16:10:18.104765 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.104774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.106088 (+ 1314us) server_negotiation.cc:407] Connection header received
0929 16:10:18.109128 (+ 3040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.109132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.109208 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.109299 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.111509 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.112188 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.114833 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.115262 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.115376 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.119880 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.119886 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.119889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.119927 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.122928 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.124067 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.124071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.124074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.124219 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.124743 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.124747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.124749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.125054 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.125169 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.126207 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.126400 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:10:18.128259 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.105009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33931 (local address 127.0.0.1:51562)
0929 16:10:18.105874 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.105935 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:18.108785 (+ 2850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:18.109580 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:18.109594 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:18.110457 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:18.111299 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.111327 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.112373 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.112382 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.114645 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.114673 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.115517 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.115527 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.115880 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.116694 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:18.116731 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:18.119597 (+ 2866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:18.123135 (+ 3538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.123143 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.123158 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.123923 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.124336 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.124344 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.124351 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.124611 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.125185 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:18.125203 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:18.125752 (+ 549us) client_negotiation.cc:769] Sending connection context
0929 16:10:18.126104 (+ 352us) client_negotiation.cc:240] Negotiation successful
0929 16:10:18.126696 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":328,"threads_started":1}
I20260929 16:10:18.130200 11113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698218, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
I20260929 16:10:18.147097 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.131117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:37742)
0929 16:10:18.131289 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.131315 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:18.131497 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:18.132281 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:18.132286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:18.132763 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:18.132907 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.132916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.134364 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.134373 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.135327 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.135340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.136224 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.136230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.136416 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.137074 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:18.137090 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:18.139287 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:18.143845 (+ 4558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.143850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.143853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.144244 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.144689 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.144693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.144695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.144830 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.145627 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:18.145665 (+ 38us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:18.145865 (+ 200us) client_negotiation.cc:769] Sending connection context
0929 16:10:18.146111 (+ 246us) client_negotiation.cc:240] Negotiation successful
0929 16:10:18.146598 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 16:10:18.147615 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.131205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37742 (local address 127.20.121.189:37581)
0929 16:10:18.131850 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.131856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.131881 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:10:18.131938 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.131944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.132131 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.132299 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.133084 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.134228 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.135488 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.136066 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.136202 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.139556 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.139576 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.139600 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.139646 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.143682 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.144398 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.144410 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.144416 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.144578 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.144939 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.144947 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.144954 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.145467 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.145684 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.146412 (+ 728us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.146995 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":320,"threads_started":1}
I20260929 16:10:18.150568 11113 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:10:18.151953 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.151359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:39416)
0929 16:10:18.151552 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.151727 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 16:10:18.152362 11064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
I20260929 16:10:18.154078 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.153672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:39432)
0929 16:10:18.153831 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.153914 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260929 16:10:18.154300 11113 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44743: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab.
Entry for principal kudu/127.20.121.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab.
I20260929 16:10:18.162199 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.188:44743
--webserver_interface=127.20.121.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab
--principal=kudu/127.20.121.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:37997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:33931,127.20.121.189:37581,127.20.121.188:44743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:10:18.203622 11113 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } attempt: 1
I20260929 16:10:18.205179 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.204608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:39448)
0929 16:10:18.204840 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.204984 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260929 16:10:18.206646 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.206204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:39462)
0929 16:10:18.206411 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.206510 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 16:10:18.206933 11113 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44743: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
I20260929 16:10:18.275192 11113 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } attempt: 2
I20260929 16:10:18.276381 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.275917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41762)
0929 16:10:18.276113 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.276213 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 16:10:18.276571 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } attempt: 5
I20260929 16:10:18.277554 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.277154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41772)
0929 16:10:18.277283 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.277391 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260929 16:10:18.277876 11113 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44743: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698217, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
I20260929 16:10:18.293186 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.277252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:44348)
0929 16:10:18.277534 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.277554 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:18.277704 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:18.278281 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:18.278286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:18.278714 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:18.278930 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.278940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.280637 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.280642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.281536 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.281549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.282738 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.282742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.282957 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.283644 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:18.283662 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:18.286122 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:18.290662 (+ 4540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.290666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.290669 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.291068 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.291506 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.291510 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.291512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.291652 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.292328 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:18.292334 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:18.292543 (+ 209us) client_negotiation.cc:769] Sending connection context
0929 16:10:18.292682 (+ 139us) client_negotiation.cc:240] Negotiation successful
0929 16:10:18.292937 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260929 16:10:18.293418 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.277380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.20.121.189:37581)
0929 16:10:18.277527 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.277535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.277644 (+ 109us) server_negotiation.cc:407] Connection header received
0929 16:10:18.277904 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.277911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.278060 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.278170 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.279127 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.280476 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.281855 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.282572 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.282712 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.286392 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.286404 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.286411 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.286470 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.290493 (+ 4023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.291205 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.291211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.291215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.291386 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.291770 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.291775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.291779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.292188 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.292340 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.292832 (+ 492us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.293081 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 16:10:18.294471 11039 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:10:18.295552 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.295164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41774)
0929 16:10:18.295336 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.295422 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 16:10:18.296526 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.296184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41788)
0929 16:10:18.296365 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.296427 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 16:10:18.296725 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44743: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
I20260929 16:10:18.347923 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } attempt: 1
I20260929 16:10:18.349058 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.348584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41802)
0929 16:10:18.348788 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.348893 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 16:10:18.350132 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.349730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41816)
0929 16:10:18.349878 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.349960 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 16:10:18.350339 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44743: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
I20260929 16:10:18.369108 11113 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } attempt: 3
I20260929 16:10:18.370334 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.369843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41832)
0929 16:10:18.370078 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.370179 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260929 16:10:18.371413 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.370989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41836)
0929 16:10:18.371219 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.371296 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260929 16:10:18.371598 11113 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44743: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
I20260929 16:10:18.418571 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } attempt: 2
I20260929 16:10:18.419996 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.419395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41848)
0929 16:10:18.419678 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.419769 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260929 16:10:18.421125 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.420719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41856)
0929 16:10:18.420935 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.421011 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260929 16:10:18.421342 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44743: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
I20260929 16:10:18.514870 11113 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } attempt: 4
I20260929 16:10:18.516186 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.515610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41862)
0929 16:10:18.515930 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.516024 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260929 16:10:18.517274 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.516924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41866)
0929 16:10:18.517070 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.517150 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 16:10:18.517472 11113 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44743: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
I20260929 16:10:18.527580 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } attempt: 3
I20260929 16:10:18.528750 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.528260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41874)
0929 16:10:18.528458 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.528555 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 16:10:18.530331 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.529576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41886)
0929 16:10:18.530088 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.530184 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369}
W20260929 16:10:18.530588 11039 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44743: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44743: connect: Connection refused (error 111)
W20260929 16:10:18.545050 11120 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 16:10:18.545492 11120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:18.545617 11120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:18.545766 11120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:18.554491 11120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:18.554649 11120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:18.554801 11120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:18.554900 11120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:18.555029 11120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:18.566666 11120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:37997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:33931,127.20.121.189:37581,127.20.121.188:44743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab
--principal=kudu/127.20.121.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.188:44743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.121.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:18.568141 11120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:18.569525 11120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:18.583293 11125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:18.583484 11126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:18.583532 11128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:18.584263 11120 server_base.cc:1034] running on GCE node
I20260929 16:10:18.585276 11120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:18.587100 11120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:18.588371 11120 hybrid_clock.cc:648] HybridClock initialized: now 1790698218588293 us; error 58 us; skew 500 ppm
I20260929 16:10:18.588811 11120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698218, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:18.596330 11120 init.cc:377] Logged in from keytab as kudu/127.20.121.188@KRBTEST.COM (short username kudu)
I20260929 16:10:18.598345 11120 webserver.cc:460] Webserver started at http://127.20.121.188:40467/ using document root <none> and password file <none>
I20260929 16:10:18.599161 11120 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:18.599269 11120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:18.599602 11120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:18.603302 11120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/data/instance:
uuid: "3d4dab74f52f4814967ec12f70283e95"
format_stamp: "Formatted at 2026-09-29 16:10:18 on dist-test-slave-x07d"
server_key: "33677efcc9ad1343cd1b6760af7bed24"
server_key_iv: "12e5e15d9c330e62946fbcb715a9c00b"
server_key_version: "encryptionkey@0"
I20260929 16:10:18.604212 11120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/wal/instance:
uuid: "3d4dab74f52f4814967ec12f70283e95"
format_stamp: "Formatted at 2026-09-29 16:10:18 on dist-test-slave-x07d"
server_key: "33677efcc9ad1343cd1b6760af7bed24"
server_key_iv: "12e5e15d9c330e62946fbcb715a9c00b"
server_key_version: "encryptionkey@0"
I20260929 16:10:18.610502 11120 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260929 16:10:18.614554 11135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:18.616291 11120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 16:10:18.616513 11120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/wal
uuid: "3d4dab74f52f4814967ec12f70283e95"
format_stamp: "Formatted at 2026-09-29 16:10:18 on dist-test-slave-x07d"
server_key: "33677efcc9ad1343cd1b6760af7bed24"
server_key_iv: "12e5e15d9c330e62946fbcb715a9c00b"
server_key_version: "encryptionkey@0"
I20260929 16:10:18.616771 11120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:18.643227 11120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:18.648005 11120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:18.648301 11120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:18.668406 11120 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.188:44743
I20260929 16:10:18.668443 11186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.188:44743 every 8 connection(s)
I20260929 16:10:18.669832 11120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
I20260929 16:10:18.673636 11187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:18.679900 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 11120
I20260929 16:10:18.680101 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790697841464038-20966-0/minicluster-data/master-2/wal/instance
I20260929 16:10:18.680632 20966 external_mini_cluster.cc:1428] Setting key 194d54d6e3873969e7314d4a8551c70e
I20260929 16:10:18.680507 11187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698218, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
I20260929 16:10:18.703145 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.683685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.20.121.190:33931)
0929 16:10:18.684209 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.684217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.684852 (+ 635us) server_negotiation.cc:407] Connection header received
0929 16:10:18.686055 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.686061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.686150 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.686257 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.688145 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.689359 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.691595 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.692302 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.692449 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.696364 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.696370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.696374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.696414 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.699477 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.700549 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.700559 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.700562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.700697 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.701178 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.701183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.701190 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.701490 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.701658 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.702481 (+ 823us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.702725 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":272,"threads_started":1}
I20260929 16:10:18.704380 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.683745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:33931 (local address 127.0.0.1:54358)
0929 16:10:18.684679 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.684732 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:18.685710 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:18.686347 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:18.686354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:18.687328 (+ 974us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:18.687925 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.687952 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.689536 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.689540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.691426 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.691442 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.692537 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.692542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.692725 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.693578 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:18.693613 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:18.696028 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:18.699640 (+ 3612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.699653 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.699667 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.700395 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.700863 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.700867 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.700869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.701059 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.701631 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:18.701639 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:18.702067 (+ 428us) client_negotiation.cc:769] Sending connection context
0929 16:10:18.702391 (+ 324us) client_negotiation.cc:240] Negotiation successful
0929 16:10:18.702925 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":508,"threads_started":1}
I20260929 16:10:18.704404 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.682764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41888 (local address 127.20.121.188:44743)
0929 16:10:18.683664 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.683675 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.683715 (+ 40us) server_negotiation.cc:407] Connection header received
0929 16:10:18.685698 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.685720 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.685997 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.686260 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.688149 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.689819 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.691594 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.692550 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.692706 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.697285 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.697301 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.697325 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.697363 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.700746 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.701569 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.701576 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.701578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.701719 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.702152 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.702157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.702159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.702538 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.702897 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.703245 (+ 348us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.703527 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":410,"threads_started":1}
I20260929 16:10:18.706298 11187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:10:18.707803 11039 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } attempt: 4
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698218, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698217, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:10:18.723573 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.707201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:37581 (local address 127.0.0.1:44354)
0929 16:10:18.707370 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.707390 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:18.707497 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:18.707839 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:18.707844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:18.708339 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:18.708546 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.708560 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.709956 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.709964 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.711388 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.711403 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.712209 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.712215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.712429 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.713022 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:18.713040 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:18.716396 (+ 3356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:18.720680 (+ 4284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.720685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.720690 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.721291 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.721743 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.721749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.721753 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.721964 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.722547 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:18.722552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:18.722721 (+ 169us) client_negotiation.cc:769] Sending connection context
0929 16:10:18.722872 (+ 151us) client_negotiation.cc:240] Negotiation successful
0929 16:10:18.723178 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":3}
I20260929 16:10:18.723596 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.707305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44354 (local address 127.20.121.189:37581)
0929 16:10:18.707482 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.707487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.707525 (+ 38us) server_negotiation.cc:407] Connection header received
0929 16:10:18.707603 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.707609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.707691 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.707807 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.708705 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.709786 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.711540 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.712072 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.712202 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.716669 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.716677 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.716684 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.716734 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.720527 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.721413 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.721421 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.721424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.721629 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.722116 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.722121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.722125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.722417 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.722588 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.722976 (+ 388us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.723205 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 16:10:18.724792 11187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:10:18.727638 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.708643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54362 (local address 127.20.121.190:33931)
0929 16:10:18.708805 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.708813 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.708829 (+ 16us) server_negotiation.cc:407] Connection header received
0929 16:10:18.708873 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.708878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.708954 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.709047 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.710688 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.711694 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.713529 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.714107 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.714296 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.719735 (+ 5439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.719741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.719744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.719781 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.724432 (+ 4651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.725318 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.725325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.725328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.725458 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.725977 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.725982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.725986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.726343 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.726636 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.726990 (+ 354us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.727234 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":42}
I20260929 16:10:18.727934 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.708417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41890)
0929 16:10:18.708582 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.708603 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:18.708722 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:18.709345 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:18.709350 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:18.709795 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:18.710063 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.710075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.711391 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.711397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.713000 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.713016 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.714060 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.714067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.714378 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.715082 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:18.715111 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:18.717626 (+ 2515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:18.723086 (+ 5460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.723093 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.723097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.724260 (+ 1163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.724812 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.724819 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.724825 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.725024 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.726739 (+ 1715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:18.726744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:18.726957 (+ 213us) client_negotiation.cc:769] Sending connection context
0929 16:10:18.727116 (+ 159us) client_negotiation.cc:240] Negotiation successful
0929 16:10:18.727447 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":58}
I20260929 16:10:18.729272 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.708886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41890 (local address 127.20.121.188:44743)
0929 16:10:18.709048 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.709053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.709070 (+ 17us) server_negotiation.cc:407] Connection header received
0929 16:10:18.709114 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.709119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.709191 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.709305 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.710200 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.711242 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.713145 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.713847 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.713982 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.718660 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.718665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.718668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.718702 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.722877 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.724448 (+ 1571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.724453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.724458 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.724628 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.725148 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.725155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.725158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.726557 (+ 1399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.728227 (+ 1670us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.728387 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.728765 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":50}
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698218, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
I20260929 16:10:18.735530 11039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50: Bootstrap starting.
I20260929 16:10:18.739297 11039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50: Neither blocks nor log segments found. Creating new log.
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
I20260929 16:10:18.740542 11039 log.cc:824] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:18.743176 11192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.725554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41902)
0929 16:10:18.725732 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.725751 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:18.725850 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:18.727173 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:18.727179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:18.727694 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:18.728577 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.728600 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.730344 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.730349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.731409 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.731422 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.732169 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.732175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.732413 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.733095 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:18.733121 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:18.736026 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:18.740293 (+ 4267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.740300 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.740305 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.740854 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.741334 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.741339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.741342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.741519 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.742287 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:18.742292 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:18.742464 (+ 172us) client_negotiation.cc:769] Sending connection context
0929 16:10:18.742624 (+ 160us) client_negotiation.cc:240] Negotiation successful
0929 16:10:18.742896 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":31}
I20260929 16:10:18.744052 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.725779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.20.121.188:44743)
0929 16:10:18.726617 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.726626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.726652 (+ 26us) server_negotiation.cc:407] Connection header received
0929 16:10:18.726706 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.726712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.726785 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.726890 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.728779 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.730179 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.731558 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.732020 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.732162 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.736284 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.736290 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.736294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.736325 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.740139 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.740975 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.740982 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.740986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.741171 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.741672 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.741676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.741679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.742143 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.742330 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.743504 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.743767 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":480,"threads_started":1}
I20260929 16:10:18.744401 11039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50: No bootstrap required, opened a new log
I20260929 16:10:18.748939 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.729474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44364 (local address 127.20.121.189:37581)
0929 16:10:18.730001 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.730039 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.730058 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:10:18.730110 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.730115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.730175 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.730280 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.731875 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.732972 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.734967 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.735509 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.735669 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.741644 (+ 5975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.741649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.741652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.741694 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.745387 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.746537 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.746544 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.746548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.746757 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.747295 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.747301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.747305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.747691 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.747977 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.748330 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.748577 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414}
I20260929 16:10:18.750401 11187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95: Bootstrap starting.
I20260929 16:10:18.754330 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:10:18.754805 11187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:18.754809 11039 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } }
I20260929 16:10:18.755208 11039 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1c63c01777f468faea535e4c6969e50, State: Initialized, Role: FOLLOWER
I20260929 16:10:18.755745 11039 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } }
I20260929 16:10:18.756213 11187 log.cc:824] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:18.756848 11195 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } } }
I20260929 16:10:18.757619 11195 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:18.757915 11014 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c1c63c01777f468faea535e4c6969e50"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54362
I20260929 16:10:18.758334 11039 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:18.758435 11014 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 16:10:18.758642 11014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:18.761720 11187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95: No bootstrap required, opened a new log
I20260929 16:10:18.762632 11014 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } }
I20260929 16:10:18.766276 11014 leader_election.cc:288] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [CANDIDATE]: Term 1 election: Requested vote from peers 8c9186e44d974aea833e1be239bb745d (127.20.121.189:37581), 3d4dab74f52f4814967ec12f70283e95 (127.20.121.188:44743)
W20260929 16:10:18.771395 11207 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 16:10:18.771670 11207 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 16:10:18.774794 11187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } }
I20260929 16:10:18.775352 11187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d4dab74f52f4814967ec12f70283e95, State: Initialized, Role: FOLLOWER
I20260929 16:10:18.775969 11187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } }
I20260929 16:10:18.777822 11210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } } }
I20260929 16:10:18.778644 11210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:18.779492 11187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:18.779539 11162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1c63c01777f468faea535e4c6969e50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3d4dab74f52f4814967ec12f70283e95"
I20260929 16:10:18.780069 11162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:18.783844 11162 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1c63c01777f468faea535e4c6969e50 in term 1.
I20260929 16:10:18.784873 10990 leader_election.cc:302] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d4dab74f52f4814967ec12f70283e95, c1c63c01777f468faea535e4c6969e50; no voters:
I20260929 16:10:18.786247 11195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [term 1 FOLLOWER]: Leader election won for term 1
W20260929 16:10:18.796468 10993 tablet.cc:2369] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:10:18.809442 11195 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [term 1 LEADER]: Becoming Leader. State: Replica: c1c63c01777f468faea535e4c6969e50, State: Running, Role: LEADER
I20260929 16:10:18.810302 11195 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } }
I20260929 16:10:18.816665 11113 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } attempt: 5
I20260929 16:10:18.773345 10972 master_runner.cc:186] Error getting master registration for 127.20.121.189:37581: OK, instance_id { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" instance_seqno: 1790698218079727 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 16:10:18.818275 10972 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 16:10:18.823868 11212 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1c63c01777f468faea535e4c6969e50. Latest consensus state: current_term: 1 leader_uuid: "c1c63c01777f468faea535e4c6969e50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } } }
I20260929 16:10:18.824373 11212 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [sys.catalog]: This master's current role is: LEADER
Sep 29 16:10:18 dist-test-slave-x07d krb5kdc[10959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698218, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
I20260929 16:10:18.834651 11226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:10:18.837141 11226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:10:18.844201 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.817291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44743 (local address 127.0.0.1:41910)
0929 16:10:18.817527 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:18.817549 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:18.817645 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:18.818355 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:18.818361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:18.818867 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:18.819095 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.819111 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.820650 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.820657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.822270 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:18.822288 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.823601 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.823610 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:18.823890 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.824712 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:18.824748 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:18.828916 (+ 4168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:18.838440 (+ 9524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.838452 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.838460 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.839455 (+ 995us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.840636 (+ 1181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:18.840643 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:18.840647 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:18.840854 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:18.842587 (+ 1733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:18.842598 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:18.842839 (+ 241us) client_negotiation.cc:769] Sending connection context
0929 16:10:18.843026 (+ 187us) client_negotiation.cc:240] Negotiation successful
0929 16:10:18.843519 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260929 16:10:18.847645 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.817349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41910 (local address 127.20.121.188:44743)
0929 16:10:18.817874 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.817880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.817897 (+ 17us) server_negotiation.cc:407] Connection header received
0929 16:10:18.817955 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.817960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.818078 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.818173 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.819282 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.820433 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.822504 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.823185 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.823322 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.829358 (+ 6036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.829371 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.829381 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.829452 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.838067 (+ 8615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.839862 (+ 1795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.839903 (+ 41us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.839911 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.840348 (+ 437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.841165 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.841174 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.841224 (+ 50us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.842280 (+ 1056us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.845613 (+ 3333us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.846000 (+ 387us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.846741 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386}
I20260929 16:10:18.855089 11113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d: Bootstrap starting.
I20260929 16:10:18.859506 11113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:18.860693 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.802997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.20.121.189:37581)
0929 16:10:18.803163 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.803169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.815367 (+ 12198us) server_negotiation.cc:407] Connection header received
0929 16:10:18.816005 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.816012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.816105 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.816182 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.829746 (+ 13564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.831448 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.838439 (+ 6991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.839248 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.839373 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.849332 (+ 9959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.849338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.849343 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.849394 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.853698 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.855845 (+ 2147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.855851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.855855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.856043 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.856856 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.856865 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.856871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.857434 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.857756 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.859146 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.859752 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 16:10:18.861487 11113 log.cc:824] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:18.874627 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.802502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.20.121.190:33931)
0929 16:10:18.802709 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.802718 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.841325 (+ 38607us) server_negotiation.cc:407] Connection header received
0929 16:10:18.844208 (+ 2883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.844215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.844299 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.844385 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.846468 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.847815 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.858347 (+ 10532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.859090 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.859213 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.865678 (+ 6465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.865689 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.865698 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.865778 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.870400 (+ 4622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.872005 (+ 1605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.872017 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.872020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.872191 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.872673 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.872678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.872681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.873099 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.873351 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.873945 (+ 594us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.874231 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 16:10:18.885632 11113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d: No bootstrap required, opened a new log
I20260929 16:10:18.891336 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:18.846653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41912 (local address 127.20.121.188:44743)
0929 16:10:18.846839 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:18.846845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:18.862637 (+ 15792us) server_negotiation.cc:407] Connection header received
0929 16:10:18.863048 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:18.863057 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:18.863173 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:18.863268 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:18.866787 (+ 3519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.868332 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.873103 (+ 4771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:18.874225 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:18.874362 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:18.881832 (+ 7470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:18.881839 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:18.881843 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:18.881897 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:18.885844 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.887406 (+ 1562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.887413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.887417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.887616 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:18.888422 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:18.888428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:18.888432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:18.888871 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:18.889165 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:18.890249 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0929 16:10:18.890708 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 16:10:18.894503 11162 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [term 1 FOLLOWER]: Refusing update from remote peer c1c63c01777f468faea535e4c6969e50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:10:18.896847 11195 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:10:18.898506 11113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } }
I20260929 16:10:18.898994 11113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9186e44d974aea833e1be239bb745d, State: Initialized, Role: FOLLOWER
I20260929 16:10:18.899576 11113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } }
W20260929 16:10:18.900704 11141 tablet.cc:2369] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:10:18.901028 11234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } } }
I20260929 16:10:18.901674 11234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:18.903214 11113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:18.903299 11088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1c63c01777f468faea535e4c6969e50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8c9186e44d974aea833e1be239bb745d"
I20260929 16:10:18.903699 11087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:18.908820 11087 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [term 1 FOLLOWER]: Refusing update from remote peer c1c63c01777f468faea535e4c6969e50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:10:18.913810 11088 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1c63c01777f468faea535e4c6969e50 in term 1.
I20260929 16:10:18.914229 11195 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 16:10:18.931535 11230 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 16:10:18.931829 11230 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 16:10:18.910215 11120 master_runner.cc:186] Error getting master registration for 127.20.121.189:37581: OK, instance_id { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" instance_seqno: 1790698218079727 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 16:10:18.938045 11120 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 16:10:18.938653 11234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1c63c01777f468faea535e4c6969e50. Latest consensus state: current_term: 1 leader_uuid: "c1c63c01777f468faea535e4c6969e50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } } }
I20260929 16:10:18.939026 11234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:18.940279 11210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1c63c01777f468faea535e4c6969e50. Latest consensus state: current_term: 1 leader_uuid: "c1c63c01777f468faea535e4c6969e50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } } }
I20260929 16:10:18.940622 11210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:18.946225 11195 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1c63c01777f468faea535e4c6969e50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } } }
I20260929 16:10:18.946724 11195 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:18.946986 11195 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1c63c01777f468faea535e4c6969e50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } } }
I20260929 16:10:18.947259 11195 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50 [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:18.948056 11226 catalog_manager.cc:1347] Generated new cluster ID: d2d09a65fd83460393e34355d782d37b
I20260929 16:10:18.948191 11226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:10:18.954962 11240 mvcc.cc:205] Tried to move back new op lower bound from 7334699904527835136 to 7334699904254812160. Current Snapshot: MvccSnapshot[applied={T|T < 7334699904527835136}]
I20260929 16:10:18.958514 11234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1c63c01777f468faea535e4c6969e50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } } }
I20260929 16:10:18.958849 11234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:18.968142 11210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1c63c01777f468faea535e4c6969e50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1c63c01777f468faea535e4c6969e50" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 33931 } } peers { permanent_uuid: "8c9186e44d974aea833e1be239bb745d" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 37581 } } peers { permanent_uuid: "3d4dab74f52f4814967ec12f70283e95" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44743 } } }
I20260929 16:10:18.968446 11210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:18.980394 11252 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 16:10:19.000176 11226 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:10:19.002285 11226 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:10:19.018618 11226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1c63c01777f468faea535e4c6969e50: Generated new TSK 0
I20260929 16:10:19.020843 11226 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 16:10:19.133457 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.120819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44384 (local address 127.20.121.189:37581)
0929 16:10:19.121059 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.121067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.130596 (+ 9529us) server_negotiation.cc:407] Connection header received
0929 16:10:19.131157 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.131164 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.131281 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.131363 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.133185 (+ 1822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44384: BlockingRecv error: recv got EOF from 127.0.0.1:44384 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260929 16:10:19.134442 11191 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.120854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.20.121.190:33931)
0929 16:10:19.121105 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.121112 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.131625 (+ 10513us) server_negotiation.cc:407] Connection header received
0929 16:10:19.132642 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.132650 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.132753 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.132836 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.134148 (+ 1312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54378: BlockingRecv error: recv got EOF from 127.0.0.1:54378 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260929 16:10:19.136197 11190 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.132238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.20.121.188:44743)
0929 16:10:19.132443 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.132449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.134417 (+ 1968us) server_negotiation.cc:407] Connection header received
0929 16:10:19.134737 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.134744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.134827 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.134902 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.135937 (+ 1035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41928: BlockingRecv error: recv got EOF from 127.0.0.1:41928 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260929 16:10:19.144773 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.142239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44388 (local address 127.20.121.189:37581)
0929 16:10:19.142465 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.142472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.143007 (+ 535us) server_negotiation.cc:407] Connection header received
0929 16:10:19.143275 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.143282 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.143426 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.143561 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.144440 (+ 879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44388: BlockingRecv error: recv got EOF from 127.0.0.1:44388 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260929 16:10:19.156464 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.149724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.20.121.189:37581)
0929 16:10:19.149929 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.149936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.154774 (+ 4838us) server_negotiation.cc:407] Connection header received
0929 16:10:19.155029 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.155036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.155155 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.155280 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.156210 (+ 930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44400: BlockingRecv error: recv got EOF from 127.0.0.1:44400 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260929 16:10:19.165158 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.162472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44416 (local address 127.20.121.189:37581)
0929 16:10:19.162714 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.162721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.163402 (+ 681us) server_negotiation.cc:407] Connection header received
0929 16:10:19.163622 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.163630 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.163747 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.163876 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.164894 (+ 1018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44416: BlockingRecv error: recv got EOF from 127.0.0.1:44416 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260929 16:10:19.179014 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.176367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.20.121.189:37581)
0929 16:10:19.176574 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.176582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.177193 (+ 611us) server_negotiation.cc:407] Connection header received
0929 16:10:19.177411 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.177418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.177519 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.177645 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.178738 (+ 1093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44428: BlockingRecv error: recv got EOF from 127.0.0.1:44428 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260929 16:10:19.193246 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.190830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.20.121.189:37581)
0929 16:10:19.191018 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.191027 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.191593 (+ 566us) server_negotiation.cc:407] Connection header received
0929 16:10:19.191797 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.191804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.191920 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.192052 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.192959 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44444: BlockingRecv error: recv got EOF from 127.0.0.1:44444 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 16:10:19.216035 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.213410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44458 (local address 127.20.121.189:37581)
0929 16:10:19.213646 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.213654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.214277 (+ 623us) server_negotiation.cc:407] Connection header received
0929 16:10:19.214509 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.214516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.214656 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.214810 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.215769 (+ 959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44458: BlockingRecv error: recv got EOF from 127.0.0.1:44458 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260929 16:10:19.254999 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.252264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44466 (local address 127.20.121.189:37581)
0929 16:10:19.252474 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.252481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.253200 (+ 719us) server_negotiation.cc:407] Connection header received
0929 16:10:19.253420 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.253427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.253533 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.253660 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.254693 (+ 1033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44466: BlockingRecv error: recv got EOF from 127.0.0.1:44466 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260929 16:10:19.330745 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.328101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.20.121.189:37581)
0929 16:10:19.328349 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.328356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.328962 (+ 606us) server_negotiation.cc:407] Connection header received
0929 16:10:19.329196 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.329204 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.329319 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.329474 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.330470 (+ 996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44480: BlockingRecv error: recv got EOF from 127.0.0.1:44480 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260929 16:10:19.464928 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.462905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44492 (local address 127.20.121.189:37581)
0929 16:10:19.463158 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.463166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.463391 (+ 225us) server_negotiation.cc:407] Connection header received
0929 16:10:19.463613 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.463623 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.463696 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.463830 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.464642 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44492: BlockingRecv error: recv got EOF from 127.0.0.1:44492 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260929 16:10:19.726799 11117 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:19.724779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.20.121.189:37581)
0929 16:10:19.725035 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:19.725042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:19.725277 (+ 235us) server_negotiation.cc:407] Connection header received
0929 16:10:19.725517 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:19.725522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:19.725596 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:19.725710 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:19.726545 (+ 835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44496: BlockingRecv error: recv got EOF from 127.0.0.1:44496 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260929 16:10:19.933933 11230 catalog_manager.cc:1259] Loaded cluster ID: d2d09a65fd83460393e34355d782d37b
I20260929 16:10:19.934069 11230 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95: loading cluster ID for follower catalog manager: success
I20260929 16:10:19.936679 11230 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95: acquiring CA information for follower catalog manager: success
I20260929 16:10:19.938874 11230 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d4dab74f52f4814967ec12f70283e95: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 16:10:19.989709 11252 catalog_manager.cc:1259] Loaded cluster ID: d2d09a65fd83460393e34355d782d37b
I20260929 16:10:19.989862 11252 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d: loading cluster ID for follower catalog manager: success
I20260929 16:10:19.992664 11252 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d: acquiring CA information for follower catalog manager: success
I20260929 16:10:19.994776 11252 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c9186e44d974aea833e1be239bb745d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 16:10:20.252701 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:20.243833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44510 (local address 127.20.121.189:37581)
0929 16:10:20.246976 (+ 3143us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:20.246984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:20.247005 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:10:20.247079 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:20.247083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:20.247282 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:20.247421 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:10:20.247883 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:20.248797 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:20.250175 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:20.250679 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:20.250821 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:20.251376 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:10:20.252026 (+ 650us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:10:20.252201 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:20.252279 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 16:10:20.252368 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2989,"thread_start_us":2813,"threads_started":1}
I20260929 16:10:20.271010 11325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:20.264285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41940 (local address 127.20.121.188:44743)
0929 16:10:20.265063 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:20.265070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:20.265091 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:10:20.265202 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:20.265206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:20.265370 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:20.265564 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:10:20.266036 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:20.266789 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:20.268089 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:20.268520 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:20.268656 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:20.269276 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:10:20.270109 (+ 833us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:10:20.270282 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:20.270453 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 16:10:20.270582 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":527,"threads_started":1}
I20260929 16:10:20.297959 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:20.277947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.20.121.189:37581)
0929 16:10:20.278160 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:20.278170 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:20.290562 (+ 12392us) server_negotiation.cc:407] Connection header received
0929 16:10:20.291096 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:20.291101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:20.291188 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:20.291256 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:10:20.292140 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:20.293234 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:20.295566 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:20.296277 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:20.296396 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:20.297067 (+ 671us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:10:20.297201 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:10:20.297271 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:20.297517 (+ 246us) server_negotiation.cc:299] Negotiation successful
0929 16:10:20.297604 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 16:10:20.299727 11332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:20.278236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.20.121.190:33931)
0929 16:10:20.290454 (+ 12218us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:20.290463 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:20.290488 (+ 25us) server_negotiation.cc:407] Connection header received
0929 16:10:20.290642 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:20.290648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:20.290883 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:20.291668 (+ 785us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:10:20.291864 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:20.293025 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:20.296014 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:20.296677 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:20.297141 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:20.297761 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:10:20.298278 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:10:20.299229 (+ 951us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:20.299301 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 16:10:20.299392 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12075,"thread_start_us":11960,"threads_started":1}
I20260929 16:10:20.300683 11325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:20.279039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41950 (local address 127.20.121.188:44743)
0929 16:10:20.279315 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:20.279323 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:20.293764 (+ 14441us) server_negotiation.cc:407] Connection header received
0929 16:10:20.294121 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:20.294127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:20.294230 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:20.294298 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:10:20.295238 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:20.296319 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:20.298619 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:20.299206 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:20.299303 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:20.299881 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:10:20.299997 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:10:20.300058 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:20.300294 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 16:10:20.300372 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 16:10:20.306079 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 10972
I20260929 16:10:20.307307 11034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:10:20.420538 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 10972
I20260929 16:10:20.442975 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 11046
I20260929 16:10:20.444223 11108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:10:20.553037 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 11046
I20260929 16:10:20.573460 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 11120
I20260929 16:10:20.574987 11182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:10:20.691268 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 11120
2026-09-29T16:10:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5008 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 16:10:20 dist-test-slave-x07d krb5kdc[11339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 16:10:20 dist-test-slave-x07d krb5kdc[11339](info): set up 2 sockets
Sep 29 16:10:20 dist-test-slave-x07d krb5kdc[11339](info): commencing operation
krb5kdc: starting...
W20260929 16:10:21.874795 20966 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.006s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/test-user.keytab.
Sep 29 16:10:22 dist-test-slave-x07d krb5kdc[11339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T16:10:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T16:10:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.121.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
Entry for principal kudu/127.20.121.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab.
I20260929 16:10:22.154946 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:42317
--webserver_interface=127.20.121.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:42317,127.20.121.189:33249,127.20.121.188:44729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 16:10:22.503014 11352 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 16:10:22.503432 11352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:22.503562 11352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:22.503700 11352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:22.511546 11352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:22.511655 11352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:22.511767 11352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:22.511859 11352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:22.511971 11352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:22.522332 11352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:42317,127.20.121.189:33249,127.20.121.188:44729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.190.keytab
--principal=kudu/127.20.121.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.190:42317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.121.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:22.523805 11352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:22.525295 11352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:22.538803 11357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:22.538702 11360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:22.538744 11358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:22.539952 11352 server_base.cc:1034] running on GCE node
I20260929 16:10:22.540813 11352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:22.542429 11352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:22.543753 11352 hybrid_clock.cc:648] HybridClock initialized: now 1790698222543613 us; error 34 us; skew 500 ppm
I20260929 16:10:22.544157 11352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:22 dist-test-slave-x07d krb5kdc[11339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698222, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:22.547955 11352 init.cc:377] Logged in from keytab as kudu/127.20.121.190@KRBTEST.COM (short username kudu)
I20260929 16:10:22.549695 11352 webserver.cc:460] Webserver started at http://127.20.121.190:39377/ using document root <none> and password file <none>
I20260929 16:10:22.550518 11352 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:22.550652 11352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:22.550992 11352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:22.554186 11352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/data/instance:
uuid: "a448fd8de2a64d68b143480aa185b13a"
format_stamp: "Formatted at 2026-09-29 16:10:22 on dist-test-slave-x07d"
server_key: "547a3e9ce58f002a9096357c3b7e5bcb"
server_key_iv: "b1f2b231da149c352cc1a78e528c2045"
server_key_version: "encryptionkey@0"
I20260929 16:10:22.555050 11352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/wal/instance:
uuid: "a448fd8de2a64d68b143480aa185b13a"
format_stamp: "Formatted at 2026-09-29 16:10:22 on dist-test-slave-x07d"
server_key: "547a3e9ce58f002a9096357c3b7e5bcb"
server_key_iv: "b1f2b231da149c352cc1a78e528c2045"
server_key_version: "encryptionkey@0"
I20260929 16:10:22.560506 11352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 16:10:22.564294 11367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:22.565682 11352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 16:10:22.565886 11352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/wal
uuid: "a448fd8de2a64d68b143480aa185b13a"
format_stamp: "Formatted at 2026-09-29 16:10:22 on dist-test-slave-x07d"
server_key: "547a3e9ce58f002a9096357c3b7e5bcb"
server_key_iv: "b1f2b231da149c352cc1a78e528c2045"
server_key_version: "encryptionkey@0"
I20260929 16:10:22.566184 11352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:22.579715 11352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:22.584498 11352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:22.584805 11352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:22.604992 11352 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.190:42317
I20260929 16:10:22.605042 11418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.190:42317 every 8 connection(s)
I20260929 16:10:22.606510 11352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/data/info.pb
I20260929 16:10:22.610759 11419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:22.613297 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 11352
I20260929 16:10:22.613507 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-0/wal/instance
I20260929 16:10:22.614084 20966 external_mini_cluster.cc:1428] Setting key 7e5014b6cfa52a00babc1f56115471e1
I20260929 16:10:22.620605 11419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:10:22 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698222, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.189@KRBTEST.COM" created.
I20260929 16:10:22.647025 11422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:22.624919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.20.121.190:42317)
0929 16:10:22.625807 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:22.625827 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:22.626302 (+ 475us) server_negotiation.cc:407] Connection header received
0929 16:10:22.628402 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:22.628408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:22.628650 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:22.628877 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:22.630249 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:22.631466 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:22.632895 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:22.633462 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:22.633673 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:22.637752 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:22.637762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:22.637777 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:22.637838 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:22.641189 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:22.642150 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:22.642158 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:22.642160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:22.642358 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:22.642950 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:22.642955 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:22.642957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:22.643637 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:22.643884 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:22.644761 (+ 877us) server_negotiation.cc:299] Negotiation successful
0929 16:10:22.645291 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":610,"thread_start_us":373,"threads_started":1}
I20260929 16:10:22.647053 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:22.624964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:42317 (local address 127.0.0.1:59942)
0929 16:10:22.626137 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:22.626179 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:22.628010 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:22.628892 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:22.628900 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:22.629640 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:22.629984 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:22.630002 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:22.631629 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:22.631635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:22.632751 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:22.632764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:22.633702 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:22.633708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:22.633890 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:22.634683 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:22.634711 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:22.637434 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:22.641380 (+ 3946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:22.641391 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:22.641402 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:22.641936 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:22.642497 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:22.642501 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:22.642504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:22.642699 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:22.643909 (+ 1210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:22.643919 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:22.644240 (+ 321us) client_negotiation.cc:769] Sending connection context
0929 16:10:22.644603 (+ 363us) client_negotiation.cc:240] Negotiation successful
0929 16:10:22.645433 (+ 830us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":230,"thread_start_us":716,"threads_started":1}
I20260929 16:10:22.650499 11419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:10:22.651935 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:22.651379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58588)
0929 16:10:22.651583 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:22.651716 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260929 16:10:22.652305 11371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
I20260929 16:10:22.654282 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:22.653821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58592)
0929 16:10:22.654004 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:22.654121 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260929 16:10:22.654526 11419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:33249: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab.
Entry for principal kudu/127.20.121.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab.
I20260929 16:10:22.683032 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.189:33249
--webserver_interface=127.20.121.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab
--principal=kudu/127.20.121.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:42317,127.20.121.189:33249,127.20.121.188:44729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:10:22.703927 11419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } attempt: 1
I20260929 16:10:22.705446 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:22.704802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58606)
0929 16:10:22.705084 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:22.705245 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260929 16:10:22.706777 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:22.706256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58620)
0929 16:10:22.706452 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:22.706563 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 16:10:22.707064 11419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:33249: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
I20260929 16:10:22.775370 11419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } attempt: 2
I20260929 16:10:22.776700 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:22.776125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58630)
0929 16:10:22.776407 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:22.776522 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260929 16:10:22.777876 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:22.777467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58634)
0929 16:10:22.777615 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:22.777714 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 16:10:22.778137 11419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:33249: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
I20260929 16:10:22.869452 11419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } attempt: 3
I20260929 16:10:22.870795 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:22.870230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58646)
0929 16:10:22.870446 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:22.870573 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260929 16:10:22.872156 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:22.871620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58648)
0929 16:10:22.871811 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:22.871934 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 16:10:22.872416 11419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:33249: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
I20260929 16:10:23.015735 11419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } attempt: 4
I20260929 16:10:23.017232 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.016529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58658)
0929 16:10:23.016890 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.017024 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260929 16:10:23.018592 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.018114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58668)
0929 16:10:23.018284 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.018412 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 16:10:23.018895 11419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.189:33249: Network error: Client connection negotiation failed: client connection to 127.20.121.189:33249: connect: Connection refused (error 111)
W20260929 16:10:23.033758 11426 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 16:10:23.034245 11426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:23.034370 11426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:23.034497 11426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:23.042858 11426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:23.042976 11426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:23.043079 11426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:23.043169 11426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:23.043289 11426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:23.054450 11426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:42317,127.20.121.189:33249,127.20.121.188:44729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.189.keytab
--principal=kudu/127.20.121.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.189:33249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.121.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:23.055910 11426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:23.057301 11426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:23.069800 11432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:23.069914 11434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:23.069885 11431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:23.070690 11426 server_base.cc:1034] running on GCE node
I20260929 16:10:23.071566 11426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:23.073146 11426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:23.074414 11426 hybrid_clock.cc:648] HybridClock initialized: now 1790698223074358 us; error 36 us; skew 500 ppm
I20260929 16:10:23.074788 11426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698223, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:23.078332 11426 init.cc:377] Logged in from keytab as kudu/127.20.121.189@KRBTEST.COM (short username kudu)
I20260929 16:10:23.080159 11426 webserver.cc:460] Webserver started at http://127.20.121.189:38009/ using document root <none> and password file <none>
I20260929 16:10:23.080976 11426 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:23.081120 11426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:23.081507 11426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:23.084888 11426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/data/instance:
uuid: "b523ba149d7942f18c003cc5c8df1379"
format_stamp: "Formatted at 2026-09-29 16:10:23 on dist-test-slave-x07d"
server_key: "3b2f2fc4e784381553eff6683507d2e4"
server_key_iv: "133c70645da1aa03da239691a25a2b76"
server_key_version: "encryptionkey@0"
I20260929 16:10:23.085843 11426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/wal/instance:
uuid: "b523ba149d7942f18c003cc5c8df1379"
format_stamp: "Formatted at 2026-09-29 16:10:23 on dist-test-slave-x07d"
server_key: "3b2f2fc4e784381553eff6683507d2e4"
server_key_iv: "133c70645da1aa03da239691a25a2b76"
server_key_version: "encryptionkey@0"
I20260929 16:10:23.091759 11426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260929 16:10:23.096035 11441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:23.097708 11426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 16:10:23.098088 11426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/wal
uuid: "b523ba149d7942f18c003cc5c8df1379"
format_stamp: "Formatted at 2026-09-29 16:10:23 on dist-test-slave-x07d"
server_key: "3b2f2fc4e784381553eff6683507d2e4"
server_key_iv: "133c70645da1aa03da239691a25a2b76"
server_key_version: "encryptionkey@0"
I20260929 16:10:23.098356 11426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:23.116791 11426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:23.121815 11426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:23.122151 11426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:23.145447 11426 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.189:33249
I20260929 16:10:23.145536 11492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.189:33249 every 8 connection(s)
I20260929 16:10:23.147012 11426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/data/info.pb
I20260929 16:10:23.150986 11493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:23.151337 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 11426
I20260929 16:10:23.151531 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-1/wal/instance
I20260929 16:10:23.151980 20966 external_mini_cluster.cc:1428] Setting key 110505eecdae123f79c5dc421f2df8ce
I20260929 16:10:23.158669 11493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698223, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.121.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.121.188@KRBTEST.COM" created.
I20260929 16:10:23.182758 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.161922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59946 (local address 127.20.121.190:42317)
0929 16:10:23.162545 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.162551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.163824 (+ 1273us) server_negotiation.cc:407] Connection header received
0929 16:10:23.166441 (+ 2617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.166449 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.166549 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.166675 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.168353 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.169135 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.170952 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.171402 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.171542 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.175715 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.175720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.175724 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.175774 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.178769 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.179940 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.179945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.179948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.180089 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.180655 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.180664 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.180670 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.181066 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.181260 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.182089 (+ 829us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.182344 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":366,"threads_started":1}
I20260929 16:10:23.183751 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.162550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:42317 (local address 127.0.0.1:59946)
0929 16:10:23.163635 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.163687 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:23.166126 (+ 2439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:23.166943 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:23.166951 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:23.167511 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:23.168110 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.168127 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.169323 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.169327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.170798 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.170812 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.171670 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.171679 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.171925 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.172678 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:23.172703 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:23.175414 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:23.179064 (+ 3650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.179077 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.179099 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.179750 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.180224 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.180230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.180242 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.180464 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.181227 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:23.181235 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:23.181647 (+ 412us) client_negotiation.cc:769] Sending connection context
0929 16:10:23.181938 (+ 291us) client_negotiation.cc:240] Negotiation successful
0929 16:10:23.182517 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":444,"threads_started":1}
I20260929 16:10:23.185916 11493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698223, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
I20260929 16:10:23.202039 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.186693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58676)
0929 16:10:23.186971 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.186990 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:23.187121 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:23.188322 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:23.188327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:23.188800 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:23.188934 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.188942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.190384 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.190390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.191241 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.191253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.192072 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.192077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.192282 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.192977 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:23.192995 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:23.195217 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:23.199039 (+ 3822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.199044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.199049 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.199436 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.199831 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.199836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.199839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.199969 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.200679 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:23.200687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:23.200864 (+ 177us) client_negotiation.cc:769] Sending connection context
0929 16:10:23.201037 (+ 173us) client_negotiation.cc:240] Negotiation successful
0929 16:10:23.201435 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":3}
I20260929 16:10:23.202118 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.187003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58676 (local address 127.20.121.189:33249)
0929 16:10:23.187857 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.187863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.187890 (+ 27us) server_negotiation.cc:407] Connection header received
0929 16:10:23.187940 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.187946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.188134 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.188333 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.189115 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.190254 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.191382 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.191923 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.192062 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.195615 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.195630 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.195645 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.195678 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.198861 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.199581 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.199589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.199591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.199700 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.200080 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.200085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.200088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.200451 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.200754 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.201209 (+ 455us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.201628 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":194,"server-negotiator.queue_time_us":622,"thread_start_us":476,"threads_started":1}
I20260929 16:10:23.204839 11493 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:10:23.206233 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.205580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50500)
0929 16:10:23.205808 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.205966 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260929 16:10:23.206530 11444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
I20260929 16:10:23.208662 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.208286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50502)
0929 16:10:23.208454 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.208533 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 16:10:23.209022 11493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44729: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.121.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab.
Entry for principal kudu/127.20.121.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab.
I20260929 16:10:23.219408 20966 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
/tmp/dist-test-taskDT3jwB/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.188:44729
--webserver_interface=127.20.121.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab
--principal=kudu/127.20.121.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.121.148:38259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.121.190:42317,127.20.121.189:33249,127.20.121.188:44729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 16:10:23.258414 11493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } attempt: 1
I20260929 16:10:23.259723 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.259159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50506)
0929 16:10:23.259382 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.259532 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 16:10:23.260839 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.260429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50522)
0929 16:10:23.260607 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.260709 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 16:10:23.261061 11493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44729: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
I20260929 16:10:23.318171 11419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } attempt: 5
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698222, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
I20260929 16:10:23.329325 11493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } attempt: 2
I20260929 16:10:23.330706 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.330084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50532)
0929 16:10:23.330376 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.330489 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 16:10:23.332015 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.331555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50548)
0929 16:10:23.331744 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.331845 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260929 16:10:23.332479 11493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44729: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
I20260929 16:10:23.334997 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.319291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58686 (local address 127.20.121.189:33249)
0929 16:10:23.319541 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.319546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.319572 (+ 26us) server_negotiation.cc:407] Connection header received
0929 16:10:23.319739 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.319744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.319834 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.319959 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.320926 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.322039 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.323308 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.324025 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.324163 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.327811 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.327821 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.327828 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.327882 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.331892 (+ 4010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.332864 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.332868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.332871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.332991 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.333414 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.333421 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.333425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.333743 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.333914 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.334403 (+ 489us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.334642 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260929 16:10:23.335016 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.319179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58686)
0929 16:10:23.319430 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.319462 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:23.319582 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:23.320018 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:23.320022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:23.320477 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:23.320723 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.320737 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.322209 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.322213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.323114 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.323139 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.324190 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.324194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.324380 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.325062 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:23.325080 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:23.327560 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:23.332139 (+ 4579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.332146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.332150 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.332702 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.333115 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.333121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.333126 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.333286 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.333879 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:23.333885 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:23.334119 (+ 234us) client_negotiation.cc:769] Sending connection context
0929 16:10:23.334288 (+ 169us) client_negotiation.cc:240] Negotiation successful
0929 16:10:23.334643 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260929 16:10:23.336179 11419 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:10:23.337313 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.336903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50558)
0929 16:10:23.337080 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.337158 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260929 16:10:23.338330 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.337917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50568)
0929 16:10:23.338114 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.338193 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 16:10:23.338600 11419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44729: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
I20260929 16:10:23.389886 11419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } attempt: 1
I20260929 16:10:23.391323 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.390795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50582)
0929 16:10:23.391043 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.391140 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260929 16:10:23.392594 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.392183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50598)
0929 16:10:23.392393 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.392474 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260929 16:10:23.392940 11419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44729: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
I20260929 16:10:23.423791 11493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } attempt: 3
I20260929 16:10:23.425100 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.424567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50614)
0929 16:10:23.424806 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.424929 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260929 16:10:23.426493 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.425954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50626)
0929 16:10:23.426252 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.426350 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
W20260929 16:10:23.426771 11493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44729: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
I20260929 16:10:23.461220 11419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } attempt: 2
I20260929 16:10:23.462625 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.462079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50628)
0929 16:10:23.462337 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.462442 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260929 16:10:23.464191 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.463614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50630)
0929 16:10:23.463903 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.463997 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260929 16:10:23.464452 11419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44729: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
I20260929 16:10:23.570133 11493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } attempt: 4
I20260929 16:10:23.570645 11419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } attempt: 3
I20260929 16:10:23.571507 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.570913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50638)
0929 16:10:23.571187 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.571298 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260929 16:10:23.571810 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.571258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50654)
0929 16:10:23.571452 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.571542 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 16:10:23.572682 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.572266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50658)
0929 16:10:23.572436 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.572524 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 16:10:23.573071 11493 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44729: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
I20260929 16:10:23.573601 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.572495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50666)
0929 16:10:23.573050 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.573237 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"threads_started":1}
W20260929 16:10:23.574090 11419 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.121.188:44729: Network error: Client connection negotiation failed: client connection to 127.20.121.188:44729: connect: Connection refused (error 111)
W20260929 16:10:23.581279 11501 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 16:10:23.581727 11501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 16:10:23.581832 11501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 16:10:23.581980 11501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 16:10:23.589653 11501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 16:10:23.589766 11501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 16:10:23.589860 11501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 16:10:23.589944 11501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 16:10:23.590067 11501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 16:10:23.600601 11501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.121.148:38259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.121.190:42317,127.20.121.189:33249,127.20.121.188:44729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/krb5kdc/kudu_127.20.121.188.keytab
--principal=kudu/127.20.121.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.121.188:44729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.121.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 14:53:52 UTC on ccycloud.quasar-tnttyx.root.comops.site
build id 4680448
ASAN enabled
I20260929 16:10:23.602126 11501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 16:10:23.603665 11501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 16:10:23.616425 11507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:23.616567 11510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 16:10:23.616441 11508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 16:10:23.617777 11501 server_base.cc:1034] running on GCE node
I20260929 16:10:23.618779 11501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 16:10:23.620316 11501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 16:10:23.621583 11501 hybrid_clock.cc:648] HybridClock initialized: now 1790698223621509 us; error 30 us; skew 500 ppm
I20260929 16:10:23.621907 11501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698223, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 16:10:23.631371 11501 init.cc:377] Logged in from keytab as kudu/127.20.121.188@KRBTEST.COM (short username kudu)
I20260929 16:10:23.633241 11501 webserver.cc:460] Webserver started at http://127.20.121.188:33739/ using document root <none> and password file <none>
I20260929 16:10:23.634105 11501 fs_manager.cc:359] Metadata directory not provided
I20260929 16:10:23.634248 11501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 16:10:23.634620 11501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 16:10:23.637843 11501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/data/instance:
uuid: "658ff8c26b74497baafe6c6d2445bf8c"
format_stamp: "Formatted at 2026-09-29 16:10:23 on dist-test-slave-x07d"
server_key: "7934c21221b2297a03ec643457896e20"
server_key_iv: "3cf6393791d0491543b712d58e056753"
server_key_version: "encryptionkey@0"
I20260929 16:10:23.638787 11501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/wal/instance:
uuid: "658ff8c26b74497baafe6c6d2445bf8c"
format_stamp: "Formatted at 2026-09-29 16:10:23 on dist-test-slave-x07d"
server_key: "7934c21221b2297a03ec643457896e20"
server_key_iv: "3cf6393791d0491543b712d58e056753"
server_key_version: "encryptionkey@0"
I20260929 16:10:23.644677 11501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260929 16:10:23.648768 11517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 16:10:23.650652 11501 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260929 16:10:23.650908 11501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/data,/tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/wal
uuid: "658ff8c26b74497baafe6c6d2445bf8c"
format_stamp: "Formatted at 2026-09-29 16:10:23 on dist-test-slave-x07d"
server_key: "7934c21221b2297a03ec643457896e20"
server_key_iv: "3cf6393791d0491543b712d58e056753"
server_key_version: "encryptionkey@0"
I20260929 16:10:23.651180 11501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 16:10:23.687038 11501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 16:10:23.691951 11501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 16:10:23.692286 11501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 16:10:23.711948 11501 rpc_server.cc:307] RPC server started. Bound to: 127.20.121.188:44729
I20260929 16:10:23.711968 11568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.121.188:44729 every 8 connection(s)
I20260929 16:10:23.713439 11501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/data/info.pb
I20260929 16:10:23.717554 11569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 16:10:23.718971 20966 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu as pid 11501
I20260929 16:10:23.719182 20966 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskDT3jwB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790697841464038-20966-0/minicluster-data/master-2/wal/instance
I20260929 16:10:23.719827 20966 external_mini_cluster.cc:1428] Setting key 531ee8380b98035029c64e1e7da3440a
I20260929 16:10:23.728482 11569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698223, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
I20260929 16:10:23.747561 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.722176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.20.121.188:44729)
0929 16:10:23.723085 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.723097 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.723144 (+ 47us) server_negotiation.cc:407] Connection header received
0929 16:10:23.724768 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.724793 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.725230 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.725600 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.727756 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.729362 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.731458 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.732761 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.734202 (+ 1441us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.738129 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.738141 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.738158 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.738216 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.742291 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.743299 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.743314 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.743327 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.743566 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.744026 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.744031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.744035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.744799 (+ 764us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.745035 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.745748 (+ 713us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.746348 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":427,"threads_started":1}
I20260929 16:10:23.751314 11419 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } attempt: 4
I20260929 16:10:23.754141 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.731700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.190:42317 (local address 127.0.0.1:59958)
0929 16:10:23.734724 (+ 3024us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.734789 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:23.735003 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:23.737123 (+ 2120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:23.737132 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:23.737788 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:23.738224 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.738236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.739832 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.739844 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.741597 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.741613 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.742557 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.742562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.742696 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.743401 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:23.743433 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:23.746457 (+ 3024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:23.750496 (+ 4039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.750508 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.750516 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.751136 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.751726 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.751731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.751738 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.751942 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.752657 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:23.752668 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:23.752955 (+ 287us) client_negotiation.cc:769] Sending connection context
0929 16:10:23.753194 (+ 239us) client_negotiation.cc:240] Negotiation successful
0929 16:10:23.753609 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2127,"thread_start_us":1891,"threads_started":1}
I20260929 16:10:23.754187 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.731817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.20.121.190:42317)
0929 16:10:23.736664 (+ 4847us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.736672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.736696 (+ 24us) server_negotiation.cc:407] Connection header received
0929 16:10:23.736762 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.736767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.736848 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.736971 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.738401 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.739518 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.741788 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.742364 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.742522 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.746695 (+ 4173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.746700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.746704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.746743 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.750333 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.751323 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.751344 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.751350 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.751563 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.752103 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.752111 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.752114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.752502 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.752717 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.753361 (+ 644us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.753672 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4705,"thread_start_us":4535,"threads_started":1}
I20260929 16:10:23.756117 11569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698222, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.190@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.190@KRBTEST.COM
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698223, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
I20260929 16:10:23.770109 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.752195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50688 (local address 127.20.121.188:44729)
0929 16:10:23.752640 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.752647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.752665 (+ 18us) server_negotiation.cc:407] Connection header received
0929 16:10:23.752715 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.752721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.752802 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.752976 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.754106 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.755123 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.757092 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.757799 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.757959 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.762462 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.762467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.762471 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.762505 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.766817 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.767910 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.767918 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.767922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.768056 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.768494 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.768501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.768507 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.768876 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.769037 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.769473 (+ 436us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.769707 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320}
I20260929 16:10:23.770149 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.752047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50688)
0929 16:10:23.752195 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.752213 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:23.752320 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:23.753017 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:23.753024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:23.753628 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:23.753881 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.753894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.755378 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.755382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.756912 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.756928 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.757977 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.757984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.758220 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.758929 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:23.758963 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:23.762253 (+ 3290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:23.766999 (+ 4746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.767010 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.767017 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.767773 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.768179 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.768185 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.768189 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.768358 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.769015 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:23.769020 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:23.769176 (+ 156us) client_negotiation.cc:769] Sending connection context
0929 16:10:23.769357 (+ 181us) client_negotiation.cc:240] Negotiation successful
0929 16:10:23.769727 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 16:10:23.772047 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.751363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59966 (local address 127.20.121.190:42317)
0929 16:10:23.752773 (+ 1410us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.752779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.752798 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:10:23.752841 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.752848 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.752924 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.754887 (+ 1963us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.755202 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.756441 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.758375 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.759154 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.759302 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.764220 (+ 4918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.764226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.764229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.764264 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.767288 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.768297 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.768306 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.768311 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.768534 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.769100 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.769107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.769111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.769598 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.769904 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.771458 (+ 1554us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.771668 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1266,"thread_start_us":1051,"threads_started":1}
I20260929 16:10:23.774159 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.757041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.20.121.189:33249)
0929 16:10:23.757223 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.757230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.757251 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:10:23.757420 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.757426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.757503 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.757612 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.758611 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.759461 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.761158 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.761733 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.761901 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.766634 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.766644 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.766651 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.766706 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.770511 (+ 3805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.771494 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.771500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.771503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.771683 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.772316 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.772321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.772324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.772817 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.772970 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.773482 (+ 512us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.773740 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 16:10:23.774237 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.756835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.189:33249 (local address 127.0.0.1:58700)
0929 16:10:23.757169 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.757188 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:23.757279 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:23.757664 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:23.757669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:23.758254 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:23.758459 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.758473 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.759617 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.759623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.760998 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.761013 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.761916 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.761923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.762160 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.762809 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:23.762835 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:23.766341 (+ 3506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:23.770686 (+ 4345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.770699 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.770703 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.771350 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.771997 (+ 647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.772001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.772003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.772182 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.773063 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:23.773066 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:23.773195 (+ 129us) client_negotiation.cc:769] Sending connection context
0929 16:10:23.773308 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 16:10:23.773853 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
I20260929 16:10:23.776446 11419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a: Bootstrap starting.
I20260929 16:10:23.777820 11569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } has no permanent_uuid. Determining permanent_uuid...
I20260929 16:10:23.780526 11419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:23.781625 11419 log.cc:824] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:23.785195 11419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a: No bootstrap required, opened a new log
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698223, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.188@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.121.189@KRBTEST.COM
I20260929 16:10:23.793000 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.778679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50698)
0929 16:10:23.779029 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.779047 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:23.779154 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:23.779503 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:23.779508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:23.779953 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:23.780099 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.780107 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.781277 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.781286 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.782545 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.782561 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.783820 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.783825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.784023 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.784736 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:23.784755 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:23.787378 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:23.790536 (+ 3158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.790543 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.790548 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.790926 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.791294 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.791299 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.791302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.791438 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.791947 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:23.791951 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:23.792099 (+ 148us) client_negotiation.cc:769] Sending connection context
0929 16:10:23.792258 (+ 159us) client_negotiation.cc:240] Negotiation successful
0929 16:10:23.792588 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221}
I20260929 16:10:23.793035 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.778826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50698 (local address 127.20.121.188:44729)
0929 16:10:23.779110 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.779116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.779133 (+ 17us) server_negotiation.cc:407] Connection header received
0929 16:10:23.779293 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.779299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.779372 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.779488 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.780252 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.781128 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.782721 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.783440 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.783586 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.787568 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.787573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.787578 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.787616 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.790393 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.791045 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.791053 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.791057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.791178 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.791539 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.791546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.791549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.791828 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.791991 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.792358 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.792717 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":142,"server-negotiator.queue_time_us":157}
I20260929 16:10:23.796831 11419 raft_consensus.cc:348] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } }
I20260929 16:10:23.797241 11419 raft_consensus.cc:729] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a448fd8de2a64d68b143480aa185b13a, State: Initialized, Role: FOLLOWER
I20260929 16:10:23.797729 11419 consensus_queue.cc:260] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } }
I20260929 16:10:23.798805 11577 sys_catalog.cc:455] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } } }
I20260929 16:10:23.799342 11577 sys_catalog.cc:458] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:23.799760 11569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c: Bootstrap starting.
I20260929 16:10:23.800434 11419 sys_catalog.cc:564] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:23.802527 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.778496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58714 (local address 127.20.121.189:33249)
0929 16:10:23.781942 (+ 3446us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.781949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.781970 (+ 21us) server_negotiation.cc:407] Connection header received
0929 16:10:23.782055 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.782060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.782149 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.782275 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.784244 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.785400 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.787307 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.787997 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.788135 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.794910 (+ 6775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.794918 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.794923 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.794971 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.798715 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.799675 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.799681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.799684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.799881 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.800346 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.800352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.800355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.800785 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.801104 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.801430 (+ 326us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.801705 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3307}
I20260929 16:10:23.805472 11569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:23.807080 11569 log.cc:824] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:23.808218 20966 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 16:10:23.811316 11569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c: No bootstrap required, opened a new log
I20260929 16:10:23.811678 11394 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a448fd8de2a64d68b143480aa185b13a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59966
I20260929 16:10:23.812093 11394 raft_consensus.cc:482] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 16:10:23.812318 11394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:23.816236 11394 raft_consensus.cc:504] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } }
I20260929 16:10:23.818988 11394 leader_election.cc:288] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [CANDIDATE]: Term 1 election: Requested vote from peers b523ba149d7942f18c003cc5c8df1379 (127.20.121.189:33249), 658ff8c26b74497baafe6c6d2445bf8c (127.20.121.188:44729)
I20260929 16:10:23.827840 11569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } }
I20260929 16:10:23.828500 11569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658ff8c26b74497baafe6c6d2445bf8c, State: Initialized, Role: FOLLOWER
I20260929 16:10:23.829252 11569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } }
W20260929 16:10:23.838099 11373 tablet.cc:2369] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:10:23.813781 11352 master_runner.cc:186] Error getting master registration for 127.20.121.189:33249: OK, instance_id { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" instance_seqno: 1790698223138539 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 16:10:23.839903 11592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } } }
I20260929 16:10:23.840662 11352 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 16:10:23.840888 11592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:23.841508 11569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:23.841530 11544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a448fd8de2a64d68b143480aa185b13a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "658ff8c26b74497baafe6c6d2445bf8c"
I20260929 16:10:23.842301 11544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [term 0 FOLLOWER]: Advancing to term 1
W20260929 16:10:23.844578 11589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 16:10:23.844990 11589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 16:10:23.846467 11544 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a448fd8de2a64d68b143480aa185b13a in term 1.
I20260929 16:10:23.847908 11370 leader_election.cc:302] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 658ff8c26b74497baafe6c6d2445bf8c, a448fd8de2a64d68b143480aa185b13a; no voters:
I20260929 16:10:23.848423 11577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 16:10:23.872277 11493 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } attempt: 5
I20260929 16:10:23.873302 11577 raft_consensus.cc:686] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [term 1 LEADER]: Becoming Leader. State: Replica: a448fd8de2a64d68b143480aa185b13a, State: Running, Role: LEADER
I20260929 16:10:23.874115 11577 consensus_queue.cc:237] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } }
I20260929 16:10:23.882563 11575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.834077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59980 (local address 127.20.121.190:42317)
0929 16:10:23.834370 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.834376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.840656 (+ 6280us) server_negotiation.cc:407] Connection header received
0929 16:10:23.842020 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.842061 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.842160 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.842245 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.846570 (+ 4325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.847414 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.854186 (+ 6772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.855330 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.855480 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.866636 (+ 11156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.866647 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.866655 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.866718 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.871892 (+ 5174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.875795 (+ 3903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.875801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.875804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.875973 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.876749 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.876755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.876758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.877281 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.877614 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.881591 (+ 3977us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.881967 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260929 16:10:23.890983 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.834773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50712 (local address 127.20.121.188:44729)
0929 16:10:23.834970 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.834976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.838712 (+ 3736us) server_negotiation.cc:407] Connection header received
0929 16:10:23.843045 (+ 4333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.843054 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.843139 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.843219 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.845580 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.846739 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.849473 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.850215 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.850325 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.862406 (+ 12081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.862414 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.862420 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.862470 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.866455 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.869282 (+ 2827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.869288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.869292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.869465 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.871136 (+ 1671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.871142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.871173 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.871842 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.876080 (+ 4238us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.890287 (+ 14207us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.890590 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 16:10:23.895430 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.841300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58726 (local address 127.20.121.189:33249)
0929 16:10:23.841491 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.841497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.851994 (+ 10497us) server_negotiation.cc:407] Connection header received
0929 16:10:23.852570 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.852580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.852694 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.852788 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.860973 (+ 8185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.862556 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.873796 (+ 11240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.874541 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.874642 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.884775 (+ 10133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.884786 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.884794 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.884862 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.890254 (+ 5392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.891534 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.891544 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.891550 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.891776 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.892517 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.892525 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.892530 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.893359 (+ 829us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.893632 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.894431 (+ 799us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.894785 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":70}
Sep 29 16:10:23 dist-test-slave-x07d krb5kdc[11339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790698223, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.121.189@KRBTEST.COM for kudu/127.20.121.188@KRBTEST.COM
I20260929 16:10:23.900504 11600 sys_catalog.cc:455] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a448fd8de2a64d68b143480aa185b13a. Latest consensus state: current_term: 1 leader_uuid: "a448fd8de2a64d68b143480aa185b13a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } } }
I20260929 16:10:23.901099 11600 sys_catalog.cc:458] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:23.906997 11607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 16:10:23.909348 11607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 16:10:23.909631 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.872939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.121.188:44729 (local address 127.0.0.1:50718)
0929 16:10:23.873334 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0929 16:10:23.873354 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 16:10:23.873469 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 16:10:23.891886 (+ 18417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 16:10:23.891892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 16:10:23.892424 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0929 16:10:23.892929 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.892948 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.894423 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.894429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.895998 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 16:10:23.896015 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.897669 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.897676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 16:10:23.897907 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.898737 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 16:10:23.898760 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 16:10:23.902119 (+ 3359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 16:10:23.906778 (+ 4659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.906787 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.906793 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.907427 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.907904 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 16:10:23.907909 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 16:10:23.907913 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 16:10:23.908072 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 16:10:23.908778 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 16:10:23.908783 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 16:10:23.908947 (+ 164us) client_negotiation.cc:769] Sending connection context
0929 16:10:23.909067 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 16:10:23.909285 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":49}
I20260929 16:10:23.910756 11572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:23.873684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.20.121.188:44729)
0929 16:10:23.891481 (+ 17797us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:23.891488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:23.891518 (+ 30us) server_negotiation.cc:407] Connection header received
0929 16:10:23.891577 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:23.891582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:23.891665 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:23.892950 (+ 1285us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:23.893126 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.894207 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.896298 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:23.897463 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:23.898291 (+ 828us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:23.902361 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 16:10:23.902367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 16:10:23.902371 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 16:10:23.902412 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 16:10:23.906611 (+ 4199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.907603 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.907610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.907615 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.907758 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 16:10:23.908272 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 16:10:23.908279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 16:10:23.908285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 16:10:23.908637 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 16:10:23.909928 (+ 1291us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:23.910078 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 16:10:23.910328 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17591,"thread_start_us":15836,"threads_started":1}
I20260929 16:10:23.916556 11493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379: Bootstrap starting.
I20260929 16:10:23.920827 11493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379: Neither blocks nor log segments found. Creating new log.
I20260929 16:10:23.922192 11493 log.cc:824] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379: Log is configured to *not* fsync() on all Append() calls
I20260929 16:10:23.933023 11544 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [term 1 FOLLOWER]: Refusing update from remote peer a448fd8de2a64d68b143480aa185b13a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:10:23.934049 11600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [LEADER]: Connected to new peer: Peer: permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:10:23.934664 11493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379: No bootstrap required, opened a new log
W20260929 16:10:23.944728 11523 tablet.cc:2369] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 16:10:23.957455 11592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a448fd8de2a64d68b143480aa185b13a. Latest consensus state: current_term: 1 leader_uuid: "a448fd8de2a64d68b143480aa185b13a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } } }
I20260929 16:10:23.957814 11592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:23.961783 11610 mvcc.cc:205] Tried to move back new op lower bound from 7334699925212725248 to 7334699924995788800. Current Snapshot: MvccSnapshot[applied={T|T < 7334699925212725248}]
I20260929 16:10:23.965917 11613 mvcc.cc:205] Tried to move back new op lower bound from 7334699925212725248 to 7334699924995788800. Current Snapshot: MvccSnapshot[applied={T|T < 7334699925212725248}]
I20260929 16:10:23.972393 11493 raft_consensus.cc:348] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } }
I20260929 16:10:23.973524 11493 raft_consensus.cc:729] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b523ba149d7942f18c003cc5c8df1379, State: Initialized, Role: FOLLOWER
I20260929 16:10:23.974732 11493 consensus_queue.cc:260] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } }
I20260929 16:10:23.977316 11577 sys_catalog.cc:455] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a448fd8de2a64d68b143480aa185b13a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } } }
I20260929 16:10:23.977659 11577 sys_catalog.cc:458] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:23.980387 11607 catalog_manager.cc:1347] Generated new cluster ID: ff9c0bd7fb5449ed8049de5a9b60e9de
I20260929 16:10:23.980505 11607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 16:10:23.983108 11592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a448fd8de2a64d68b143480aa185b13a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } } }
I20260929 16:10:23.983393 11592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:23.985006 11468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a448fd8de2a64d68b143480aa185b13a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b523ba149d7942f18c003cc5c8df1379"
I20260929 16:10:23.984958 11493 sys_catalog.cc:564] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 16:10:23.985533 11468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [term 0 FOLLOWER]: Advancing to term 1
I20260929 16:10:23.989241 11468 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a448fd8de2a64d68b143480aa185b13a in term 1.
I20260929 16:10:23.983322 11618 sys_catalog.cc:455] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } } }
I20260929 16:10:23.990173 11618 sys_catalog.cc:458] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:23.990630 11467 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [term 1 FOLLOWER]: Refusing update from remote peer a448fd8de2a64d68b143480aa185b13a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 16:10:23.992316 11600 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 16:10:24.012051 11607 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 16:10:24.014945 11607 catalog_manager.cc:1504] Loading token signing keys...
I20260929 16:10:24.017717 11501 master_runner.cc:186] Error getting master registration for 127.20.121.189:33249: OK, instance_id { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" instance_seqno: 1790698223138539 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 16:10:24.028090 11501 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 16:10:24.027890 11618 sys_catalog.cc:455] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a448fd8de2a64d68b143480aa185b13a. Latest consensus state: current_term: 1 leader_uuid: "a448fd8de2a64d68b143480aa185b13a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } } }
I20260929 16:10:24.028247 11618 sys_catalog.cc:458] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:24.028774 11629 sys_catalog.cc:455] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a448fd8de2a64d68b143480aa185b13a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } } }
I20260929 16:10:24.029187 11629 sys_catalog.cc:458] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 16:10:24.030787 11577 sys_catalog.cc:455] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a448fd8de2a64d68b143480aa185b13a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a448fd8de2a64d68b143480aa185b13a" member_type: VOTER last_known_addr { host: "127.20.121.190" port: 42317 } } peers { permanent_uuid: "b523ba149d7942f18c003cc5c8df1379" member_type: VOTER last_known_addr { host: "127.20.121.189" port: 33249 } } peers { permanent_uuid: "658ff8c26b74497baafe6c6d2445bf8c" member_type: VOTER last_known_addr { host: "127.20.121.188" port: 44729 } } }
I20260929 16:10:24.031113 11577 sys_catalog.cc:458] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a [sys.catalog]: This master's current role is: LEADER
I20260929 16:10:24.032740 11625 catalog_manager.cc:1259] Loaded cluster ID: ff9c0bd7fb5449ed8049de5a9b60e9de
I20260929 16:10:24.032853 11625 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c: loading cluster ID for follower catalog manager: success
I20260929 16:10:24.042829 11607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a448fd8de2a64d68b143480aa185b13a: Generated new TSK 0
I20260929 16:10:24.044049 11607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 16:10:24.045595 11625 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c: acquiring CA information for follower catalog manager: success
I20260929 16:10:24.047025 11636 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 16:10:24.048460 11625 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 658ff8c26b74497baafe6c6d2445bf8c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 16:10:24.124236 11572 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.112490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50720 (local address 127.20.121.188:44729)
0929 16:10:24.112729 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.112738 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.120063 (+ 7325us) server_negotiation.cc:407] Connection header received
0929 16:10:24.122320 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.122329 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.122453 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.122558 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.123907 (+ 1349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50720: BlockingRecv error: recv got EOF from 127.0.0.1:50720 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260929 16:10:24.127537 11575 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.111926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59990 (local address 127.20.121.190:42317)
0929 16:10:24.112219 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.112226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.123328 (+ 11102us) server_negotiation.cc:407] Connection header received
0929 16:10:24.123803 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.123812 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.123907 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.123980 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.127210 (+ 3230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59990: BlockingRecv error: recv got EOF from 127.0.0.1:59990 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260929 16:10:24.127839 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.120791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.20.121.189:33249)
0929 16:10:24.121201 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.121210 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.125770 (+ 4560us) server_negotiation.cc:407] Connection header received
0929 16:10:24.126491 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.126497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.126579 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.126652 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.127626 (+ 974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58742: BlockingRecv error: recv got EOF from 127.0.0.1:58742 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
W20260929 16:10:24.140125 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.133634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58746 (local address 127.20.121.189:33249)
0929 16:10:24.133814 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.133819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.138608 (+ 4789us) server_negotiation.cc:407] Connection header received
0929 16:10:24.138878 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.138883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.138985 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.139110 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.139918 (+ 808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58746: BlockingRecv error: recv got EOF from 127.0.0.1:58746 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 16:10:24.170418 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.156797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.20.121.189:33249)
0929 16:10:24.157019 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.157028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.168089 (+ 11061us) server_negotiation.cc:407] Connection header received
0929 16:10:24.168642 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.168649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.168782 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.168871 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.170177 (+ 1306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58756: BlockingRecv error: recv got EOF from 127.0.0.1:58756 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260929 16:10:24.179533 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.177184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.20.121.189:33249)
0929 16:10:24.177432 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.177443 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.177990 (+ 547us) server_negotiation.cc:407] Connection header received
0929 16:10:24.178264 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.178270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.178348 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.178474 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.179259 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58766: BlockingRecv error: recv got EOF from 127.0.0.1:58766 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260929 16:10:24.188925 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.186734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58782 (local address 127.20.121.189:33249)
0929 16:10:24.187005 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.187014 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.187542 (+ 528us) server_negotiation.cc:407] Connection header received
0929 16:10:24.187758 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.187769 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.187854 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.187967 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.188689 (+ 722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58782: BlockingRecv error: recv got EOF from 127.0.0.1:58782 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260929 16:10:24.205494 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.203003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58794 (local address 127.20.121.189:33249)
0929 16:10:24.203239 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.203250 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.203838 (+ 588us) server_negotiation.cc:407] Connection header received
0929 16:10:24.204091 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.204104 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.204240 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.204371 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.205226 (+ 855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58794: BlockingRecv error: recv got EOF from 127.0.0.1:58794 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260929 16:10:24.227172 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.224874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.20.121.189:33249)
0929 16:10:24.225149 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.225158 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.225707 (+ 549us) server_negotiation.cc:407] Connection header received
0929 16:10:24.225936 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.225940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.226077 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.226199 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.226935 (+ 736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58800: BlockingRecv error: recv got EOF from 127.0.0.1:58800 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260929 16:10:24.264955 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.262641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58806 (local address 127.20.121.189:33249)
0929 16:10:24.262887 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.262896 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.263451 (+ 555us) server_negotiation.cc:407] Connection header received
0929 16:10:24.263696 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.263701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.263780 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.263906 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.264693 (+ 787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58806: BlockingRecv error: recv got EOF from 127.0.0.1:58806 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260929 16:10:24.334964 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.332468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.20.121.189:33249)
0929 16:10:24.332716 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.332726 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.333329 (+ 603us) server_negotiation.cc:407] Connection header received
0929 16:10:24.333574 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.333579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.333660 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.333797 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.334717 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58818: BlockingRecv error: recv got EOF from 127.0.0.1:58818 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260929 16:10:24.474813 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.471759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.20.121.189:33249)
0929 16:10:24.472013 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.472023 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.472926 (+ 903us) server_negotiation.cc:407] Connection header received
0929 16:10:24.473193 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.473200 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.473346 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.473485 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.474521 (+ 1036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58830: BlockingRecv error: recv got EOF from 127.0.0.1:58830 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260929 16:10:24.736218 11498 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 16:10:24.734383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58834 (local address 127.20.121.189:33249)
0929 16:10:24.734627 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:24.734631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:24.734798 (+ 167us) server_negotiation.cc:407] Connection header received
0929 16:10:24.735025 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:24.735030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:24.735113 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:24.735222 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 16:10:24.736002 (+ 780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58834: BlockingRecv error: recv got EOF from 127.0.0.1:58834 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 16:10:25.051914 11636 catalog_manager.cc:1259] Loaded cluster ID: ff9c0bd7fb5449ed8049de5a9b60e9de
I20260929 16:10:25.052141 11636 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379: loading cluster ID for follower catalog manager: success
I20260929 16:10:25.055440 11636 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379: acquiring CA information for follower catalog manager: success
I20260929 16:10:25.058061 11636 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b523ba149d7942f18c003cc5c8df1379: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 16:10:25.261117 11704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:25.252864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.20.121.189:33249)
0929 16:10:25.253589 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:25.253596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:25.254070 (+ 474us) server_negotiation.cc:407] Connection header received
0929 16:10:25.254338 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:25.254345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:25.254540 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:25.254701 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:10:25.255432 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:25.256778 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:25.258243 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:25.258953 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:25.259092 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:25.259647 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:10:25.260409 (+ 762us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:10:25.260564 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:25.260622 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 16:10:25.260721 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":386,"threads_started":1}
I20260929 16:10:25.273844 11710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:25.267544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50730 (local address 127.20.121.188:44729)
0929 16:10:25.268223 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:25.268228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:25.268251 (+ 23us) server_negotiation.cc:407] Connection header received
0929 16:10:25.268395 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:25.268399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:25.268558 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:25.268751 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:10:25.269276 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:25.270225 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:25.271636 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:25.272036 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:25.272174 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:25.272687 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:10:25.273220 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:10:25.273392 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:25.273453 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 16:10:25.273534 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":444,"threads_started":1}
I20260929 16:10:25.299378 11710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:25.280151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.20.121.188:44729)
0929 16:10:25.280387 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:25.280396 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:25.291653 (+ 11257us) server_negotiation.cc:407] Connection header received
0929 16:10:25.291968 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:25.291976 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:25.292099 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:25.292173 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:10:25.292986 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:25.294154 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:25.297053 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:25.297674 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:25.297794 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:25.298533 (+ 739us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:10:25.298668 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:10:25.298750 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:25.298992 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 16:10:25.299080 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 16:10:25.301345 11704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:25.292195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.20.121.189:33249)
0929 16:10:25.292403 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:25.292409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:25.294666 (+ 2257us) server_negotiation.cc:407] Connection header received
0929 16:10:25.294864 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:25.294871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:25.294979 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:25.295067 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:10:25.295812 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:25.296971 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:25.299084 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:25.299743 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:25.299842 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:25.300504 (+ 662us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:10:25.300616 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:10:25.300683 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:25.300924 (+ 241us) server_negotiation.cc:299] Negotiation successful
0929 16:10:25.301011 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 16:10:25.301373 11715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 16:10:25.279858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59992 (local address 127.20.121.190:42317)
0929 16:10:25.292484 (+ 12626us) server_negotiation.cc:206] Beginning negotiation
0929 16:10:25.292489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 16:10:25.292508 (+ 19us) server_negotiation.cc:407] Connection header received
0929 16:10:25.292625 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 16:10:25.292630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 16:10:25.292995 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 16:10:25.293225 (+ 230us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 16:10:25.295041 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:25.296137 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:25.298415 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 16:10:25.299069 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 16:10:25.299163 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 16:10:25.300171 (+ 1008us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 16:10:25.300633 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 16:10:25.300718 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 16:10:25.300924 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 16:10:25.301011 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12508,"thread_start_us":13523,"threads_started":1}
I20260929 16:10:25.307173 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 11352
I20260929 16:10:25.308521 11414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:10:25.422786 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 11352
W20260929 16:10:25.443838 11520 connection.cc:535] server connection from 127.0.0.1:50688 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 16:10:25.444257 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 11426
I20260929 16:10:25.445437 11488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:10:25.553807 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 11426
I20260929 16:10:25.574306 20966 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu pid 11501
I20260929 16:10:25.575522 11564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 16:10:25.684921 20966 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskDT3jwB/build/asan/bin/kudu with pid 11501
2026-09-29T16:10:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4999 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10008 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (384185 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST